2023-03-09 01:53:33,564 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-09 01:53:33,837 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb 2023-03-09 01:53:33,850 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-09 01:53:33,850 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/cluster_c7ae5eb2-da64-f932-cf16-ebb509eb1321, deleteOnExit=true 2023-03-09 01:53:33,850 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-09 01:53:33,851 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/test.cache.data in system properties and HBase conf 2023-03-09 01:53:33,851 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/hadoop.tmp.dir in system properties and HBase conf 2023-03-09 01:53:33,852 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/hadoop.log.dir in system properties and HBase conf 2023-03-09 01:53:33,852 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-09 01:53:33,852 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-09 01:53:33,853 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-09 01:53:33,963 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-09 01:53:34,383 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-09 01:53:34,388 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-09 01:53:34,388 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-09 01:53:34,389 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-09 01:53:34,389 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-09 01:53:34,389 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-09 01:53:34,389 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-09 01:53:34,390 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-09 01:53:34,390 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-09 01:53:34,391 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-09 01:53:34,391 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/nfs.dump.dir in system properties and HBase conf 2023-03-09 01:53:34,391 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/java.io.tmpdir in system properties and HBase conf 2023-03-09 01:53:34,392 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-09 01:53:34,392 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-09 01:53:34,393 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-09 01:53:35,007 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-09 01:53:35,011 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-09 01:53:35,326 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-09 01:53:35,527 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-09 01:53:35,549 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-09 01:53:35,595 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-09 01:53:35,628 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/java.io.tmpdir/Jetty_localhost_43755_hdfs____wdiin6/webapp 2023-03-09 01:53:35,777 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43755 2023-03-09 01:53:35,795 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-09 01:53:35,795 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-09 01:53:36,345 WARN [Listener at localhost/45187] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-09 01:53:36,446 WARN [Listener at localhost/45187] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-09 01:53:36,470 WARN [Listener at localhost/45187] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-09 01:53:36,478 INFO [Listener at localhost/45187] log.Slf4jLog(67): jetty-6.1.26 2023-03-09 01:53:36,485 INFO [Listener at localhost/45187] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/java.io.tmpdir/Jetty_localhost_39281_datanode____yycr1o/webapp 2023-03-09 01:53:36,593 INFO [Listener at localhost/45187] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39281 2023-03-09 01:53:36,986 WARN [Listener at localhost/33973] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-09 01:53:37,580 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x37f1997d699c3f75: Processing first storage report for DS-31e319b1-befe-4be8-895d-3d7dab642687 from datanode 80cf466a-c239-40a6-88c4-b2c3dde97f78 2023-03-09 01:53:37,582 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x37f1997d699c3f75: from storage DS-31e319b1-befe-4be8-895d-3d7dab642687 node DatanodeRegistration(127.0.0.1:36171, datanodeUuid=80cf466a-c239-40a6-88c4-b2c3dde97f78, infoPort=43415, infoSecurePort=0, ipcPort=33973, storageInfo=lv=-57;cid=testClusterID;nsid=280978619;c=1678326815090), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-09 01:53:37,582 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x37f1997d699c3f75: Processing first storage report for DS-d8b33789-508d-4a9c-9ac2-b81adfd5f7a6 from datanode 80cf466a-c239-40a6-88c4-b2c3dde97f78 2023-03-09 01:53:37,582 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x37f1997d699c3f75: from storage DS-d8b33789-508d-4a9c-9ac2-b81adfd5f7a6 node DatanodeRegistration(127.0.0.1:36171, datanodeUuid=80cf466a-c239-40a6-88c4-b2c3dde97f78, infoPort=43415, infoSecurePort=0, ipcPort=33973, storageInfo=lv=-57;cid=testClusterID;nsid=280978619;c=1678326815090), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-09 01:53:37,612 DEBUG [Listener at localhost/33973] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb 2023-03-09 01:53:37,679 INFO [Listener at localhost/33973] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/cluster_c7ae5eb2-da64-f932-cf16-ebb509eb1321/zookeeper_0, clientPort=50548, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/cluster_c7ae5eb2-da64-f932-cf16-ebb509eb1321/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/cluster_c7ae5eb2-da64-f932-cf16-ebb509eb1321/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-09 01:53:37,693 INFO [Listener at localhost/33973] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=50548 2023-03-09 01:53:37,700 INFO [Listener at localhost/33973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:53:37,702 INFO [Listener at localhost/33973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:53:38,370 INFO [Listener at localhost/33973] util.FSUtils(471): Created version file at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf with version=8 2023-03-09 01:53:38,371 INFO [Listener at localhost/33973] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/hbase-staging 2023-03-09 01:53:38,694 INFO [Listener at localhost/33973] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-09 01:53:39,177 INFO [Listener at localhost/33973] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-09 01:53:39,209 INFO [Listener at localhost/33973] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-09 01:53:39,209 INFO [Listener at localhost/33973] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-09 01:53:39,209 INFO [Listener at localhost/33973] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-09 01:53:39,210 INFO [Listener at localhost/33973] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-09 01:53:39,210 INFO [Listener at localhost/33973] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-09 01:53:39,379 INFO [Listener at localhost/33973] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-09 01:53:39,453 DEBUG [Listener at localhost/33973] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-09 01:53:39,547 INFO [Listener at localhost/33973] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39575 2023-03-09 01:53:39,558 INFO [Listener at localhost/33973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:53:39,561 INFO [Listener at localhost/33973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:53:39,583 INFO [Listener at localhost/33973] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39575 connecting to ZooKeeper ensemble=127.0.0.1:50548 2023-03-09 01:53:39,626 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:395750x0, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-09 01:53:39,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39575-0x100c6facd450000 connected 2023-03-09 01:53:39,668 DEBUG [Listener at localhost/33973] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-09 01:53:39,669 DEBUG [Listener at localhost/33973] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-09 01:53:39,672 DEBUG [Listener at localhost/33973] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-09 01:53:39,680 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39575 2023-03-09 01:53:39,680 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39575 2023-03-09 01:53:39,681 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39575 2023-03-09 01:53:39,682 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39575 2023-03-09 01:53:39,682 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39575 2023-03-09 01:53:39,689 INFO [Listener at localhost/33973] master.HMaster(444): hbase.rootdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf, hbase.cluster.distributed=false 2023-03-09 01:53:39,758 INFO [Listener at localhost/33973] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-09 01:53:39,758 INFO [Listener at localhost/33973] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-09 01:53:39,759 INFO [Listener at localhost/33973] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-09 01:53:39,759 INFO [Listener at localhost/33973] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-09 01:53:39,759 INFO [Listener at localhost/33973] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-09 01:53:39,759 INFO [Listener at localhost/33973] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-09 01:53:39,764 INFO [Listener at localhost/33973] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-09 01:53:39,767 INFO [Listener at localhost/33973] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46511 2023-03-09 01:53:39,769 INFO [Listener at localhost/33973] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-09 01:53:39,775 DEBUG [Listener at localhost/33973] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-09 01:53:39,776 INFO [Listener at localhost/33973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:53:39,779 INFO [Listener at localhost/33973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:53:39,781 INFO [Listener at localhost/33973] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46511 connecting to ZooKeeper ensemble=127.0.0.1:50548 2023-03-09 01:53:39,784 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:465110x0, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-09 01:53:39,785 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46511-0x100c6facd450001 connected 2023-03-09 01:53:39,785 DEBUG [Listener at localhost/33973] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-09 01:53:39,787 DEBUG [Listener at localhost/33973] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-09 01:53:39,788 DEBUG [Listener at localhost/33973] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-09 01:53:39,788 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46511 2023-03-09 01:53:39,788 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46511 2023-03-09 01:53:39,789 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46511 2023-03-09 01:53:39,789 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46511 2023-03-09 01:53:39,790 DEBUG [Listener at localhost/33973] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46511 2023-03-09 01:53:39,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:53:39,801 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-09 01:53:39,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:53:39,821 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-09 01:53:39,821 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-09 01:53:39,821 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:53:39,822 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-09 01:53:39,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39575,1678326818523 from backup master directory 2023-03-09 01:53:39,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-09 01:53:39,828 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:53:39,828 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-09 01:53:39,829 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-09 01:53:39,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:53:39,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-09 01:53:39,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-09 01:53:39,928 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/hbase.id with ID: f00e9d2a-cde4-47a5-9415-4c4bd13e0d15 2023-03-09 01:53:39,972 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:53:39,988 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:53:40,033 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x33053f4c to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:53:40,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b7e011a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:53:40,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-09 01:53:40,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-09 01:53:40,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-09 01:53:40,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-09 01:53:40,125 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-09 01:53:40,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-09 01:53:40,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-09 01:53:40,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store-tmp 2023-03-09 01:53:40,596 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:40,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-09 01:53:40,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-09 01:53:40,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-09 01:53:40,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-09 01:53:40,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-09 01:53:40,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-09 01:53:40,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-09 01:53:40,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/WALs/jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:53:40,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39575%2C1678326818523, suffix=, logDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/WALs/jenkins-hbase4.apache.org,39575,1678326818523, archiveDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/oldWALs, maxLogs=10 2023-03-09 01:53:40,684 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36171,DS-31e319b1-befe-4be8-895d-3d7dab642687,DISK] 2023-03-09 01:53:40,694 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-09 01:53:40,756 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/WALs/jenkins-hbase4.apache.org,39575,1678326818523/jenkins-hbase4.apache.org%2C39575%2C1678326818523.1678326820641 2023-03-09 01:53:40,756 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36171,DS-31e319b1-befe-4be8-895d-3d7dab642687,DISK]] 2023-03-09 01:53:40,757 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:53:40,757 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:40,761 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-09 01:53:40,762 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-09 01:53:40,829 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-09 01:53:40,837 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-09 01:53:40,862 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-09 01:53:40,875 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:40,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-09 01:53:40,883 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-09 01:53:40,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-09 01:53:40,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:53:40,911 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70223151, jitterRate=0.046406492590904236}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:53:40,911 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-09 01:53:40,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-09 01:53:40,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-09 01:53:40,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-09 01:53:40,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-09 01:53:40,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-09 01:53:40,973 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 36 msec 2023-03-09 01:53:40,973 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-09 01:53:41,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-09 01:53:41,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-09 01:53:41,035 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-09 01:53:41,040 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-09 01:53:41,042 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-09 01:53:41,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-09 01:53:41,055 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-09 01:53:41,059 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:53:41,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-09 01:53:41,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-09 01:53:41,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-09 01:53:41,088 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-09 01:53:41,088 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-09 01:53:41,088 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:53:41,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39575,1678326818523, sessionid=0x100c6facd450000, setting cluster-up flag (Was=false) 2023-03-09 01:53:41,109 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:53:41,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-09 01:53:41,120 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:53:41,127 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:53:41,134 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-09 01:53:41,136 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:53:41,139 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp 2023-03-09 01:53:41,196 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(952): ClusterId : f00e9d2a-cde4-47a5-9415-4c4bd13e0d15 2023-03-09 01:53:41,203 DEBUG [RS:0;jenkins-hbase4:46511] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-09 01:53:41,211 DEBUG [RS:0;jenkins-hbase4:46511] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-09 01:53:41,211 DEBUG [RS:0;jenkins-hbase4:46511] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-09 01:53:41,219 DEBUG [RS:0;jenkins-hbase4:46511] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-09 01:53:41,221 DEBUG [RS:0;jenkins-hbase4:46511] zookeeper.ReadOnlyZKClient(139): Connect 0x177d308c to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:53:41,231 DEBUG [RS:0;jenkins-hbase4:46511] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@19bacc13, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:53:41,234 DEBUG [RS:0;jenkins-hbase4:46511] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d855bfc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-09 01:53:41,266 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46511 2023-03-09 01:53:41,271 INFO [RS:0;jenkins-hbase4:46511] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-09 01:53:41,271 INFO [RS:0;jenkins-hbase4:46511] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-09 01:53:41,271 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1023): About to register with Master. 2023-03-09 01:53:41,275 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,39575,1678326818523 with isa=jenkins-hbase4.apache.org/172.31.14.131:46511, startcode=1678326819757 2023-03-09 01:53:41,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-09 01:53:41,295 DEBUG [RS:0;jenkins-hbase4:46511] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-09 01:53:41,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-09 01:53:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-09 01:53:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-09 01:53:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-09 01:53:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-09 01:53:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-09 01:53:41,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1678326851309 2023-03-09 01:53:41,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-09 01:53:41,314 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-09 01:53:41,314 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-09 01:53:41,322 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-09 01:53:41,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-09 01:53:41,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-09 01:53:41,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-09 01:53:41,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-09 01:53:41,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-09 01:53:41,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-09 01:53:41,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-09 01:53:41,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-09 01:53:41,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-09 01:53:41,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-09 01:53:41,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678326821353,5,FailOnTimeoutGroup] 2023-03-09 01:53:41,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678326821353,5,FailOnTimeoutGroup] 2023-03-09 01:53:41,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-09 01:53:41,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,370 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46111, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-09 01:53:41,372 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-09 01:53:41,374 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-09 01:53:41,374 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf 2023-03-09 01:53:41,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:41,402 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:41,405 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-09 01:53:41,409 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/info 2023-03-09 01:53:41,410 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-09 01:53:41,412 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:41,412 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-09 01:53:41,417 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf 2023-03-09 01:53:41,418 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:45187 2023-03-09 01:53:41,418 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-09 01:53:41,419 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/rep_barrier 2023-03-09 01:53:41,420 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-09 01:53:41,421 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:41,422 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-09 01:53:41,425 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-09 01:53:41,426 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/table 2023-03-09 01:53:41,426 DEBUG [RS:0;jenkins-hbase4:46511] zookeeper.ZKUtil(162): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:41,426 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-09 01:53:41,427 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:41,426 WARN [RS:0;jenkins-hbase4:46511] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-09 01:53:41,429 INFO [RS:0;jenkins-hbase4:46511] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-09 01:53:41,429 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1947): logDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/WALs/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:41,430 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740 2023-03-09 01:53:41,430 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46511,1678326819757] 2023-03-09 01:53:41,431 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740 2023-03-09 01:53:41,437 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-09 01:53:41,439 DEBUG [RS:0;jenkins-hbase4:46511] zookeeper.ZKUtil(162): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:41,440 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-09 01:53:41,444 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:53:41,445 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70354660, jitterRate=0.04836612939834595}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-09 01:53:41,446 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-09 01:53:41,446 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-09 01:53:41,446 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-09 01:53:41,446 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-09 01:53:41,446 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-09 01:53:41,446 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-09 01:53:41,448 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-09 01:53:41,448 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-09 01:53:41,453 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-09 01:53:41,454 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-09 01:53:41,454 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-09 01:53:41,462 INFO [RS:0;jenkins-hbase4:46511] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-09 01:53:41,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-09 01:53:41,480 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-09 01:53:41,484 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-09 01:53:41,488 INFO [RS:0;jenkins-hbase4:46511] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-09 01:53:41,492 INFO [RS:0;jenkins-hbase4:46511] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:53:41,493 INFO [RS:0;jenkins-hbase4:46511] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,496 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-09 01:53:41,505 INFO [RS:0;jenkins-hbase4:46511] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,506 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,506 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,506 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,506 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,506 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,507 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-09 01:53:41,507 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,507 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,507 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,507 DEBUG [RS:0;jenkins-hbase4:46511] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-09 01:53:41,510 INFO [RS:0;jenkins-hbase4:46511] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,510 INFO [RS:0;jenkins-hbase4:46511] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,510 INFO [RS:0;jenkins-hbase4:46511] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,534 INFO [RS:0;jenkins-hbase4:46511] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-09 01:53:41,537 INFO [RS:0;jenkins-hbase4:46511] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46511,1678326819757-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:41,557 INFO [RS:0;jenkins-hbase4:46511] regionserver.Replication(203): jenkins-hbase4.apache.org,46511,1678326819757 started 2023-03-09 01:53:41,557 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,46511,1678326819757, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46511, sessionid=0x100c6facd450001 2023-03-09 01:53:41,558 DEBUG [RS:0;jenkins-hbase4:46511] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-09 01:53:41,558 DEBUG [RS:0;jenkins-hbase4:46511] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:41,558 DEBUG [RS:0;jenkins-hbase4:46511] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46511,1678326819757' 2023-03-09 01:53:41,558 DEBUG [RS:0;jenkins-hbase4:46511] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:41,559 DEBUG [RS:0;jenkins-hbase4:46511] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:41,559 DEBUG [RS:0;jenkins-hbase4:46511] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-09 01:53:41,559 DEBUG [RS:0;jenkins-hbase4:46511] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-09 01:53:41,559 DEBUG [RS:0;jenkins-hbase4:46511] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:41,559 DEBUG [RS:0;jenkins-hbase4:46511] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46511,1678326819757' 2023-03-09 01:53:41,559 DEBUG [RS:0;jenkins-hbase4:46511] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-09 01:53:41,560 DEBUG [RS:0;jenkins-hbase4:46511] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-09 01:53:41,560 DEBUG [RS:0;jenkins-hbase4:46511] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-09 01:53:41,560 INFO [RS:0;jenkins-hbase4:46511] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-09 01:53:41,561 INFO [RS:0;jenkins-hbase4:46511] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-09 01:53:41,637 DEBUG [jenkins-hbase4:39575] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-09 01:53:41,639 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46511,1678326819757, state=OPENING 2023-03-09 01:53:41,647 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-09 01:53:41,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:53:41,650 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-09 01:53:41,654 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:53:41,677 INFO [RS:0;jenkins-hbase4:46511] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46511%2C1678326819757, suffix=, logDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/WALs/jenkins-hbase4.apache.org,46511,1678326819757, archiveDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/oldWALs, maxLogs=32 2023-03-09 01:53:41,700 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36171,DS-31e319b1-befe-4be8-895d-3d7dab642687,DISK] 2023-03-09 01:53:41,704 INFO [RS:0;jenkins-hbase4:46511] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/WALs/jenkins-hbase4.apache.org,46511,1678326819757/jenkins-hbase4.apache.org%2C46511%2C1678326819757.1678326821681 2023-03-09 01:53:41,704 DEBUG [RS:0;jenkins-hbase4:46511] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36171,DS-31e319b1-befe-4be8-895d-3d7dab642687,DISK]] 2023-03-09 01:53:41,846 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:41,848 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-09 01:53:41,852 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34710, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-09 01:53:41,864 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-09 01:53:41,865 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-09 01:53:41,872 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46511%2C1678326819757.meta, suffix=.meta, logDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/WALs/jenkins-hbase4.apache.org,46511,1678326819757, archiveDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/oldWALs, maxLogs=32 2023-03-09 01:53:41,890 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36171,DS-31e319b1-befe-4be8-895d-3d7dab642687,DISK] 2023-03-09 01:53:41,897 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/WALs/jenkins-hbase4.apache.org,46511,1678326819757/jenkins-hbase4.apache.org%2C46511%2C1678326819757.meta.1678326821874.meta 2023-03-09 01:53:41,898 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36171,DS-31e319b1-befe-4be8-895d-3d7dab642687,DISK]] 2023-03-09 01:53:41,898 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:53:41,900 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-09 01:53:41,919 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-09 01:53:41,924 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-09 01:53:41,930 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-09 01:53:41,930 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:41,930 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-09 01:53:41,930 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-09 01:53:41,934 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-09 01:53:41,936 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/info 2023-03-09 01:53:41,936 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/info 2023-03-09 01:53:41,936 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-09 01:53:41,937 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:41,937 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-09 01:53:41,939 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/rep_barrier 2023-03-09 01:53:41,939 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/rep_barrier 2023-03-09 01:53:41,939 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-09 01:53:41,940 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:41,940 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-09 01:53:41,942 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/table 2023-03-09 01:53:41,942 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/table 2023-03-09 01:53:41,942 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-09 01:53:41,943 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:41,945 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740 2023-03-09 01:53:41,947 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740 2023-03-09 01:53:41,950 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-09 01:53:41,953 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-09 01:53:41,954 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66952366, jitterRate=-0.0023320019245147705}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-09 01:53:41,954 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-09 01:53:41,965 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1678326821837 2023-03-09 01:53:41,982 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-09 01:53:41,983 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-09 01:53:41,983 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46511,1678326819757, state=OPEN 2023-03-09 01:53:41,986 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-09 01:53:41,986 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-09 01:53:41,991 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-09 01:53:41,991 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46511,1678326819757 in 332 msec 2023-03-09 01:53:41,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-09 01:53:41,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 527 msec 2023-03-09 01:53:42,001 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 802 msec 2023-03-09 01:53:42,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1678326822001, completionTime=-1 2023-03-09 01:53:42,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-09 01:53:42,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-03-09 01:53:42,072 DEBUG [hconnection-0x27f12041-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:53:42,076 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34718, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:53:42,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-03-09 01:53:42,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1678326882098 2023-03-09 01:53:42,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1678326942098 2023-03-09 01:53:42,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 96 msec 2023-03-09 01:53:42,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39575,1678326818523-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:42,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39575,1678326818523-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:42,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39575,1678326818523-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:42,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39575, period=300000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:42,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-09 01:53:42,131 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-09 01:53:42,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-09 01:53:42,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-09 01:53:42,149 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-09 01:53:42,151 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:53:42,154 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:53:42,176 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:42,179 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655 empty. 2023-03-09 01:53:42,179 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:42,179 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-09 01:53:42,601 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-09 01:53:42,603 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0ea7b302e0cce81de94432b7e794c655, NAME => 'hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:53:42,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:42,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 0ea7b302e0cce81de94432b7e794c655, disabling compactions & flushes 2023-03-09 01:53:42,617 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:53:42,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:53:42,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. after waiting 0 ms 2023-03-09 01:53:42,618 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:53:42,618 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:53:42,618 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 0ea7b302e0cce81de94432b7e794c655: 2023-03-09 01:53:42,623 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:53:42,639 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678326822625"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326822625"}]},"ts":"1678326822625"} 2023-03-09 01:53:42,667 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:53:42,670 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:53:42,676 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326822670"}]},"ts":"1678326822670"} 2023-03-09 01:53:42,680 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-09 01:53:42,691 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=0ea7b302e0cce81de94432b7e794c655, ASSIGN}] 2023-03-09 01:53:42,695 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=0ea7b302e0cce81de94432b7e794c655, ASSIGN 2023-03-09 01:53:42,697 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=0ea7b302e0cce81de94432b7e794c655, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:53:42,848 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=0ea7b302e0cce81de94432b7e794c655, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:42,848 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678326822848"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326822848"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326822848"}]},"ts":"1678326822848"} 2023-03-09 01:53:42,853 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 0ea7b302e0cce81de94432b7e794c655, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:53:43,013 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:53:43,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0ea7b302e0cce81de94432b7e794c655, NAME => 'hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:53:43,014 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:43,015 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:43,015 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:43,015 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:43,017 INFO [StoreOpener-0ea7b302e0cce81de94432b7e794c655-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:43,019 DEBUG [StoreOpener-0ea7b302e0cce81de94432b7e794c655-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655/info 2023-03-09 01:53:43,019 DEBUG [StoreOpener-0ea7b302e0cce81de94432b7e794c655-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655/info 2023-03-09 01:53:43,020 INFO [StoreOpener-0ea7b302e0cce81de94432b7e794c655-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ea7b302e0cce81de94432b7e794c655 columnFamilyName info 2023-03-09 01:53:43,020 INFO [StoreOpener-0ea7b302e0cce81de94432b7e794c655-1] regionserver.HStore(310): Store=0ea7b302e0cce81de94432b7e794c655/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:43,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:43,023 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:43,027 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:53:43,031 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:53:43,032 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0ea7b302e0cce81de94432b7e794c655; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67176866, jitterRate=0.0010133087635040283}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:53:43,032 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0ea7b302e0cce81de94432b7e794c655: 2023-03-09 01:53:43,034 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655., pid=6, masterSystemTime=1678326823006 2023-03-09 01:53:43,038 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:53:43,038 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:53:43,039 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=0ea7b302e0cce81de94432b7e794c655, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:43,040 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1678326823039"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326823039"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326823039"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326823039"}]},"ts":"1678326823039"} 2023-03-09 01:53:43,047 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-09 01:53:43,047 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 0ea7b302e0cce81de94432b7e794c655, server=jenkins-hbase4.apache.org,46511,1678326819757 in 190 msec 2023-03-09 01:53:43,051 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-09 01:53:43,051 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=0ea7b302e0cce81de94432b7e794c655, ASSIGN in 356 msec 2023-03-09 01:53:43,052 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:53:43,053 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326823052"}]},"ts":"1678326823052"} 2023-03-09 01:53:43,055 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-09 01:53:43,060 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:53:43,062 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 919 msec 2023-03-09 01:53:43,152 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-09 01:53:43,154 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-09 01:53:43,154 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:53:43,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-09 01:53:43,210 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-09 01:53:43,217 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 34 msec 2023-03-09 01:53:43,228 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-09 01:53:43,241 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-09 01:53:43,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-03-09 01:53:43,259 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-09 01:53:43,262 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-09 01:53:43,265 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.436sec 2023-03-09 01:53:43,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-09 01:53:43,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-09 01:53:43,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-09 01:53:43,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39575,1678326818523-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-09 01:53:43,272 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39575,1678326818523-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-09 01:53:43,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-09 01:53:43,302 DEBUG [Listener at localhost/33973] zookeeper.ReadOnlyZKClient(139): Connect 0x449b853a to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:53:43,308 DEBUG [Listener at localhost/33973] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4dbd1c1c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:53:43,323 DEBUG [hconnection-0x5831e476-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:53:43,337 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34726, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:53:43,350 INFO [Listener at localhost/33973] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:53:43,351 DEBUG [Listener at localhost/33973] zookeeper.ReadOnlyZKClient(139): Connect 0x143ae868 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:53:43,357 DEBUG [Listener at localhost/33973] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c173c99, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:53:43,357 INFO [Listener at localhost/33973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:53:43,381 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=353, ProcessCount=173, AvailableMemoryMB=8355 2023-03-09 01:53:43,392 DEBUG [Listener at localhost/33973] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:53:43,396 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43794, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:53:43,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-09 01:53:43,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:43,417 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:53:43,419 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:53:43,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-09 01:53:43,423 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,424 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de empty. 2023-03-09 01:53:43,426 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,426 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-09 01:53:43,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-09 01:53:43,457 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-09 01:53:43,460 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 80524e4cfd7024a7ff50a654d546b0de, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:53:43,475 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:43,475 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 80524e4cfd7024a7ff50a654d546b0de, disabling compactions & flushes 2023-03-09 01:53:43,475 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:43,475 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:43,475 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. after waiting 0 ms 2023-03-09 01:53:43,475 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:43,476 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:43,476 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:43,480 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:53:43,482 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678326823482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326823482"}]},"ts":"1678326823482"} 2023-03-09 01:53:43,485 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:53:43,487 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:53:43,487 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326823487"}]},"ts":"1678326823487"} 2023-03-09 01:53:43,490 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-09 01:53:43,494 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=80524e4cfd7024a7ff50a654d546b0de, ASSIGN}] 2023-03-09 01:53:43,497 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=80524e4cfd7024a7ff50a654d546b0de, ASSIGN 2023-03-09 01:53:43,499 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=80524e4cfd7024a7ff50a654d546b0de, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:53:43,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-09 01:53:43,650 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=80524e4cfd7024a7ff50a654d546b0de, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:43,650 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678326823650"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326823650"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326823650"}]},"ts":"1678326823650"} 2023-03-09 01:53:43,654 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 80524e4cfd7024a7ff50a654d546b0de, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:53:43,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-09 01:53:43,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:43,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 80524e4cfd7024a7ff50a654d546b0de, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:53:43,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:43,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,818 INFO [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,820 DEBUG [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:43,820 DEBUG [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:43,823 INFO [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80524e4cfd7024a7ff50a654d546b0de columnFamilyName family1 2023-03-09 01:53:43,826 INFO [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] regionserver.HStore(310): Store=80524e4cfd7024a7ff50a654d546b0de/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:43,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:43,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:53:43,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 80524e4cfd7024a7ff50a654d546b0de; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66303363, jitterRate=-0.012002900242805481}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:53:43,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:43,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de., pid=11, masterSystemTime=1678326823807 2023-03-09 01:53:43,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:43,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:43,848 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=80524e4cfd7024a7ff50a654d546b0de, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:43,848 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678326823847"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326823847"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326823847"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326823847"}]},"ts":"1678326823847"} 2023-03-09 01:53:43,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-09 01:53:43,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 80524e4cfd7024a7ff50a654d546b0de, server=jenkins-hbase4.apache.org,46511,1678326819757 in 197 msec 2023-03-09 01:53:43,859 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-09 01:53:43,859 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=80524e4cfd7024a7ff50a654d546b0de, ASSIGN in 361 msec 2023-03-09 01:53:43,861 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:53:43,861 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326823861"}]},"ts":"1678326823861"} 2023-03-09 01:53:43,864 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-09 01:53:43,868 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:53:43,871 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 456 msec 2023-03-09 01:53:44,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-09 01:53:44,048 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-09 01:53:44,065 DEBUG [hconnection-0x6cda1dbe-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:53:44,069 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47108, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:53:44,229 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:44,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,237 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:44,237 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:44,238 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:44,238 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:44,238 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,239 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,240 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:44,240 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:44,241 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,241 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,242 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,242 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,245 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:44,245 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:44,245 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:44,246 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:44,246 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:44,246 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:44,246 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:44,247 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-09 01:53:44,798 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130dba6cf58a39d40bfa2d93192d78b24c9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130dba6cf58a39d40bfa2d93192d78b24c9 2023-03-09 01:53:44,813 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/fe1a1e8b364d4f44b2df6b459f509f5d 2023-03-09 01:53:44,823 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/fe1a1e8b364d4f44b2df6b459f509f5d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/fe1a1e8b364d4f44b2df6b459f509f5d 2023-03-09 01:53:44,832 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/fe1a1e8b364d4f44b2df6b459f509f5d, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-09 01:53:44,835 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 588ms, sequenceid=5, compaction requested=false 2023-03-09 01:53:44,837 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:44,839 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:44,839 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:44,840 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:44,840 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:44,840 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,840 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:44,840 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:44,844 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,844 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:44,844 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,844 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:44,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:44,846 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:44,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@500d4214[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:44,846 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,848 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,848 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:44,849 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:44,849 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:44,849 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,849 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:44,851 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:44,852 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:44,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:44,855 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:44,855 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:44,855 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:44,855 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,855 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:44,859 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,859 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:44,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:44,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,859 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:44,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:44,860 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,861 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,868 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,869 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:44,869 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,869 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:44,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:44,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:44,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:44,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:44,869 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:44,869 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,870 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,870 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,870 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:44,871 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:44,871 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:44,971 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:44,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:44,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:44,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:44,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,992 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:44,992 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:44,992 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:44,992 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:44,993 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,993 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,995 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:44,995 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:44,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:44,996 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:44,996 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:44,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:44,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-09 01:53:44,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:44,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:44,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:44,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:44,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:44,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:44,998 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:44,998 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:44,999 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-09 01:53:45,427 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511286e08558ba1144c9d9ccee1f0041e0b7e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511286e08558ba1144c9d9ccee1f0041e0b7e 2023-03-09 01:53:45,439 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/87f12ffc1f3f4ca3879d13886453931e 2023-03-09 01:53:45,449 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/87f12ffc1f3f4ca3879d13886453931e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/87f12ffc1f3f4ca3879d13886453931e 2023-03-09 01:53:45,458 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/87f12ffc1f3f4ca3879d13886453931e, entries=1, sequenceid=9, filesize=5.0 K 2023-03-09 01:53:45,460 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 461ms, sequenceid=9, compaction requested=false 2023-03-09 01:53:45,460 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:45,461 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:45,461 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:45,461 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:45,461 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,461 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:45,461 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:45,463 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,463 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:45,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:45,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:45,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:45,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:45,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:45,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76a19e2b[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:45,466 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:45,466 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,468 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,468 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:45,468 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:45,468 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:45,469 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,469 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:45,471 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:45,471 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,471 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:45,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:45,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:45,476 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:45,476 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:45,476 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:45,476 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,476 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:45,484 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,484 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:45,484 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:45,485 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,488 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,493 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,493 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:45,493 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:45,493 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,493 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:45,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:45,493 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,493 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,493 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:45,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-09 01:53:45,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:45,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:45,493 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:45,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:45,494 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,495 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:45,495 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:45,595 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:45,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:45,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:45,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:45,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,608 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:45,608 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:45,608 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:45,608 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:45,609 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,609 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,610 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,610 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:45,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:45,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:45,611 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:45,611 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:45,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:45,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-09 01:53:45,612 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:45,612 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:45,612 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:45,613 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:45,613 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:45,613 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:45,613 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:45,613 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:45,613 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:53:46,041 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512015fb8740e6ea445b981af318617117411 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512015fb8740e6ea445b981af318617117411 2023-03-09 01:53:46,453 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/5a5a58532f95448482a07c5f7ee337b6 2023-03-09 01:53:46,461 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/5a5a58532f95448482a07c5f7ee337b6 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/5a5a58532f95448482a07c5f7ee337b6 2023-03-09 01:53:46,470 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/5a5a58532f95448482a07c5f7ee337b6, entries=1, sequenceid=13, filesize=5.0 K 2023-03-09 01:53:46,472 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 859ms, sequenceid=13, compaction requested=true 2023-03-09 01:53:46,472 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:46,472 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:46,472 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:46,473 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:46,473 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,473 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:46,473 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:46,476 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,476 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:46,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:46,476 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,477 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:46,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:46,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:46,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:46,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:46,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47f80aea[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:46,478 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:46,478 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,481 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,481 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:46,481 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:46,481 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:46,481 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,481 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:46,483 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:46,483 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,483 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:46,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:46,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:46,483 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:46,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:46,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:46,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:46,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:46,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:46,486 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:46,487 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:46,487 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:46,487 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,487 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:46,488 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,488 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:46,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,489 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:46,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:46,489 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:46,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,491 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:46,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,495 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,495 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:46,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:46,495 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:46,495 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:46,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:46,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:46,495 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:46,496 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,496 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,496 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,496 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,496 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,496 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-09 01:53:46,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:46,496 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:46,497 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:46,597 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:46,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:46,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:46,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:46,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,609 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:46,609 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:46,609 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:46,609 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:46,610 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,610 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,612 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:46,612 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:46,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:46,612 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:46,612 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:46,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,613 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:46,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,613 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,614 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-09 01:53:46,614 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:46,614 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:46,615 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:46,615 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:53:47,040 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050fe29f704da647ec8a46d3ce565c0ff2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050fe29f704da647ec8a46d3ce565c0ff2 2023-03-09 01:53:47,449 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:53:47,453 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/539392aeac254883a70057b9820ad8d3 2023-03-09 01:53:47,466 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/539392aeac254883a70057b9820ad8d3 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/539392aeac254883a70057b9820ad8d3 2023-03-09 01:53:47,478 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/539392aeac254883a70057b9820ad8d3, entries=1, sequenceid=17, filesize=5.0 K 2023-03-09 01:53:47,480 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 865ms, sequenceid=17, compaction requested=true 2023-03-09 01:53:47,480 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:47,480 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:47,480 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:47,481 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:47,481 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,481 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:47,481 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:47,484 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,484 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:47,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:47,485 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,485 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:47,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:47,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65379d2b[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:47,488 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:47,488 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,490 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,490 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:47,490 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:47,490 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:47,490 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,490 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:47,492 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:47,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:47,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:47,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:47,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:47,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:47,497 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:47,497 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:47,497 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:47,497 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,497 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:47,504 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,504 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:47,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,504 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:47,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:47,505 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:47,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,507 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,540 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,540 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:47,540 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:47,540 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,540 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:47,540 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:47,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:47,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:47,540 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,544 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,544 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,544 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:47,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-09 01:53:47,544 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:47,544 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:47,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:47,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:47,549 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-09 01:53:47,549 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-09 01:53:47,645 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:47,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:47,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:47,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:47,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,659 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:47,659 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:47,659 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:47,659 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:47,660 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,660 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,662 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:47,662 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:47,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:47,662 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:47,662 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:47,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:47,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,664 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,664 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:47,664 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:47,664 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:47,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:47,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:47,665 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:47,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:47,665 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:47,665 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:53:47,700 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228854007390dda4aaa9dba025242911547 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228854007390dda4aaa9dba025242911547 2023-03-09 01:53:48,127 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/15d7c63277784856b34ef90986794771 2023-03-09 01:53:48,140 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/15d7c63277784856b34ef90986794771 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/15d7c63277784856b34ef90986794771 2023-03-09 01:53:48,150 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/15d7c63277784856b34ef90986794771, entries=1, sequenceid=21, filesize=5.0 K 2023-03-09 01:53:48,151 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 486ms, sequenceid=21, compaction requested=true 2023-03-09 01:53:48,151 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:48,151 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:48,151 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:48,152 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:48,152 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,152 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:48,152 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,158 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:48,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:48,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:48,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:48,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:48,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:48,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f906e7b[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:48,161 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:48,161 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,163 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:48,163 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,163 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:48,165 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,165 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:48,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:48,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:48,165 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:48,165 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:48,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:48,168 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:48,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:48,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:48,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,168 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:48,170 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,170 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:48,170 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:48,171 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,173 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,177 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,177 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,177 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,177 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,177 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,177 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,177 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,177 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:48,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:48,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:48,178 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-09 01:53:48,179 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:48,179 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:48,184 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:48,184 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:48,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:48,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:48,279 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:48,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:48,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:48,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:48,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,292 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:48,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:48,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:48,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:48,293 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,293 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,294 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,294 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:48,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:48,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:48,295 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,295 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:48,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:48,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,296 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-09 01:53:48,296 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:48,297 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:48,297 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:48,297 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:48,297 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:48,297 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:48,297 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:53:48,688 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-09 01:53:48,689 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-09 01:53:48,721 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ca3bdcd9b1954092a483ab573323474e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231ca3bdcd9b1954092a483ab573323474e 2023-03-09 01:53:48,731 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/d9ccc2a087a1452f8cda8bb094b0642e 2023-03-09 01:53:48,739 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/d9ccc2a087a1452f8cda8bb094b0642e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/d9ccc2a087a1452f8cda8bb094b0642e 2023-03-09 01:53:48,747 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/d9ccc2a087a1452f8cda8bb094b0642e, entries=1, sequenceid=25, filesize=5.0 K 2023-03-09 01:53:48,749 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 452ms, sequenceid=25, compaction requested=true 2023-03-09 01:53:48,749 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:48,749 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:48,749 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:48,749 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:48,749 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,749 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:48,749 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:48,752 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,752 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:48,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:48,753 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,753 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:48,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:48,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@253ae364[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:48,755 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:48,755 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,757 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:48,757 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:48,757 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:48,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:48,758 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:48,758 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,758 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:48,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:48,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:48,758 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:48,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:48,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:48,762 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:48,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:48,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:48,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,762 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:48,764 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,764 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:48,764 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:48,765 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:48,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:48,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:48,766 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:48,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:48,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:48,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:48,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:48,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:48,771 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:48,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,772 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,772 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-09 01:53:48,772 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:48,772 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:48,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:48,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:48,872 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:48,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:48,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:48,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:48,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,886 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:48,886 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:48,886 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:48,886 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:48,887 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,887 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,889 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:48,889 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:48,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:48,889 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:48,889 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:48,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:48,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,890 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,891 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-09 01:53:48,891 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:48,891 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:48,891 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:48,891 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:48,892 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:48,892 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:48,892 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:48,892 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:48,892 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:53:49,316 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601017c627ce4b13e466095a572007f62c59a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601017c627ce4b13e466095a572007f62c59a 2023-03-09 01:53:49,725 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/b8584d43b27441c891d5dcea807d5e28 2023-03-09 01:53:49,734 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/b8584d43b27441c891d5dcea807d5e28 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/b8584d43b27441c891d5dcea807d5e28 2023-03-09 01:53:49,743 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/b8584d43b27441c891d5dcea807d5e28, entries=1, sequenceid=29, filesize=5.0 K 2023-03-09 01:53:49,744 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 852ms, sequenceid=29, compaction requested=true 2023-03-09 01:53:49,744 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:49,745 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:49,745 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:49,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:49,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:49,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:49,748 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,748 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:49,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:49,748 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,748 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:49,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:49,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:49,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:49,750 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:49,750 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22921273[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:49,750 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:49,750 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,752 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,753 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:49,753 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:49,753 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:49,753 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,753 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:49,754 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:49,754 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:49,754 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,754 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:49,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:49,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:49,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:49,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:49,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:49,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:49,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:49,757 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:49,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:49,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:49,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,757 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:49,759 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,759 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:49,759 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:49,760 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:49,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,761 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:49,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:49,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:49,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:49,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:49,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,771 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:49,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:49,771 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:49,772 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,772 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,772 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,772 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:49,772 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:49,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:49,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:49,872 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:49,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:49,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:49,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:49,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,884 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:49,884 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:49,884 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:49,884 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:49,885 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,885 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,887 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,887 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:49,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:49,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:49,887 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:49,887 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:49,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:49,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-09 01:53:49,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:49,889 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:49,889 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:49,889 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:49,889 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:49,890 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:49,890 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:49,890 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:49,890 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:53:49,910 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103358fb5e3d72546aba605c6a1b9721d71 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103358fb5e3d72546aba605c6a1b9721d71 2023-03-09 01:53:50,319 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/4976ce0c3ded4290950ffac896f7540e 2023-03-09 01:53:50,328 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/4976ce0c3ded4290950ffac896f7540e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/4976ce0c3ded4290950ffac896f7540e 2023-03-09 01:53:50,336 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/4976ce0c3ded4290950ffac896f7540e, entries=1, sequenceid=33, filesize=5.0 K 2023-03-09 01:53:50,337 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 447ms, sequenceid=33, compaction requested=true 2023-03-09 01:53:50,337 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:50,337 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:50,337 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:50,338 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:50,338 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,338 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:50,338 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:50,340 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,340 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:50,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:50,340 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,340 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:50,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:50,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:50,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:50,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:50,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b950dcf[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:50,342 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:50,342 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,343 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,344 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:50,344 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:50,344 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:50,344 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,344 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:50,347 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:50,347 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,347 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:50,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:50,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:50,347 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:50,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:50,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:50,350 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:50,350 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:50,350 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:50,350 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,350 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:50,353 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,353 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:50,353 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:50,353 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:50,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:50,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:50,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,355 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,358 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:50,358 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,358 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,358 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:50,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-09 01:53:50,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:50,358 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:50,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:50,359 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:50,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:50,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:50,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:50,359 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,360 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,360 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:50,360 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:50,360 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,360 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,460 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:50,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:50,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:50,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:50,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,472 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:50,472 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:50,473 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:50,473 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:50,473 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,473 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,475 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:50,475 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:50,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:50,476 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,476 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:50,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:50,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,476 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-09 01:53:50,482 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:50,483 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:50,483 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:50,483 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:50,483 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:50,483 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:50,483 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:50,484 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:50,484 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:53:50,913 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090775ca55b03ac14fbabbe4ded77fe8f074 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015090775ca55b03ac14fbabbe4ded77fe8f074 2023-03-09 01:53:51,323 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/83d6453e8be74a65b7eab840b7986300 2023-03-09 01:53:51,334 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/83d6453e8be74a65b7eab840b7986300 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/83d6453e8be74a65b7eab840b7986300 2023-03-09 01:53:51,343 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/83d6453e8be74a65b7eab840b7986300, entries=1, sequenceid=37, filesize=5.0 K 2023-03-09 01:53:51,345 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 861ms, sequenceid=37, compaction requested=true 2023-03-09 01:53:51,345 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:51,345 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:51,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:51,347 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:51,347 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,347 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:51,347 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:51,349 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,349 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:51,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:51,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:51,350 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,350 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:51,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:51,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:51,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:51,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@272a4770[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:51,352 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:51,352 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,353 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,354 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:51,354 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:51,354 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:51,354 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,354 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:51,355 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,355 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:51,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:51,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:51,356 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:51,356 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:51,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:51,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:51,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:51,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:51,360 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:51,360 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:51,360 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:51,360 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,360 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:51,363 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,363 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:51,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,363 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:51,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:51,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:51,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:51,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,364 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:51,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:51,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,366 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:51,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,371 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:51,371 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:51,371 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,371 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,371 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:51,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:51,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:51,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:51,371 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:51,372 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,372 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,372 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,372 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-09 01:53:51,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:51,372 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:51,373 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:51,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:51,473 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:51,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:51,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:51,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:51,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,492 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:51,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:51,493 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-09 01:53:51,493 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-09 01:53:51,493 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,494 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,496 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,496 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:51,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:51,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:51,496 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:51,496 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:51,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,497 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:51,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,507 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-09 01:53:51,507 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:51,507 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-09 01:53:51,507 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:51,508 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-09 01:53:51,509 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:51,509 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:51,509 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:51,509 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. started... 2023-03-09 01:53:51,509 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80524e4cfd7024a7ff50a654d546b0de 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:53:51,937 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120293b261419274fdf8a6cb684dd99f320 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120293b261419274fdf8a6cb684dd99f320 2023-03-09 01:53:52,369 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/36a965be5831477c9acf89877e4c89b3 2023-03-09 01:53:52,377 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/36a965be5831477c9acf89877e4c89b3 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/36a965be5831477c9acf89877e4c89b3 2023-03-09 01:53:52,384 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/36a965be5831477c9acf89877e4c89b3, entries=1, sequenceid=41, filesize=5.0 K 2023-03-09 01:53:52,385 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80524e4cfd7024a7ff50a654d546b0de in 876ms, sequenceid=41, compaction requested=true 2023-03-09 01:53:52,386 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:52,386 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:52,386 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:52,386 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:52,386 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,386 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-09 01:53:52,386 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-09 01:53:52,388 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,388 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:52,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:52,389 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:52,389 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:52,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:52,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:52,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-09 01:53:52,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55f9b132[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:52,390 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-09 01:53:52,390 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,391 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,392 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-09 01:53:52,392 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-09 01:53:52,392 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,392 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:52,392 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:52,396 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,396 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-09 01:53:52,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:52,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:52,396 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:52,396 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-09 01:53:52,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:52,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:52,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:52,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:52,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-09 01:53:52,401 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-09 01:53:52,401 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:52,401 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:52,401 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,401 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:52,403 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,403 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:52,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:52,403 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:52,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,404 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:52,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:52,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:52,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:52,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,405 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:52,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,408 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:52,408 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:52,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:52,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-09 01:53:52,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:52,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:52,408 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:52,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:52,409 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:52,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-09 01:53:52,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:52,409 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:52,411 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,411 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,411 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,411 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:52,411 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,411 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,411 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,509 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:52,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:52,516 DEBUG [Listener at localhost/33973] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-09 01:53:52,518 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43800, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-09 01:53:52,519 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:52,535 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-09 01:53:52,535 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-09 01:53:52,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:53:52,537 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327432537 2023-03-09 01:53:52,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:53:52,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-09 01:53:52,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:52,543 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x41e2de52 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:53:52,548 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77a64d3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:53:52,549 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,433 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038bb71b0c7bd8438287756cbf6c3cca8a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038bb71b0c7bd8438287756cbf6c3cca8a 2023-03-09 01:53:53,434 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:53,453 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:53:53,455 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52910, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:53:53,467 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:53:53,469 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44366, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:53:53,509 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/de5654aa162e44748bead3f524382bba first=Optional[r3] last=Optional[r6] 2023-03-09 01:53:53,525 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/de5654aa162e44748bead3f524382bba}] 2023-03-09 01:53:53,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/de5654aa162e44748bead3f524382bba for inclusion in 80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:53,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-09 01:53:53,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:53:53,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:53,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/de5654aa162e44748bead3f524382bba to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ddrpss3hqmlqig9cni7kvuqbemhij7rgeijp2gcj1fpbc51ct76rmgjk3na7qm8a/family1/de5654aa162e44748bead3f524382bba 2023-03-09 01:53:53,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ddrpss3hqmlqig9cni7kvuqbemhij7rgeijp2gcj1fpbc51ct76rmgjk3na7qm8a/family1/de5654aa162e44748bead3f524382bba as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/1df6d40bab3f4200aad4410c1f53be26_SeqId_44_ 2023-03-09 01:53:53,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ddrpss3hqmlqig9cni7kvuqbemhij7rgeijp2gcj1fpbc51ct76rmgjk3na7qm8a/family1/de5654aa162e44748bead3f524382bba into 80524e4cfd7024a7ff50a654d546b0de/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/1df6d40bab3f4200aad4410c1f53be26_SeqId_44_ - updating store file list. 2023-03-09 01:53:53,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/1df6d40bab3f4200aad4410c1f53be26_SeqId_44_ into 80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:53,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ddrpss3hqmlqig9cni7kvuqbemhij7rgeijp2gcj1fpbc51ct76rmgjk3na7qm8a/family1/de5654aa162e44748bead3f524382bba into 80524e4cfd7024a7ff50a654d546b0de/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/1df6d40bab3f4200aad4410c1f53be26_SeqId_44_) 2023-03-09 01:53:53,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ddrpss3hqmlqig9cni7kvuqbemhij7rgeijp2gcj1fpbc51ct76rmgjk3na7qm8a/family1/de5654aa162e44748bead3f524382bba 2023-03-09 01:53:53,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-09 01:53:53,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-09 01:53:53,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 80524e4cfd7024a7ff50a654d546b0de/family1 is initiating minor compaction 2023-03-09 01:53:53,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@53d9ba57; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:53:53,582 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 80524e4cfd7024a7ff50a654d546b0de/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:53,583 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/87f12ffc1f3f4ca3879d13886453931e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/5a5a58532f95448482a07c5f7ee337b6, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/539392aeac254883a70057b9820ad8d3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/15d7c63277784856b34ef90986794771, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/d9ccc2a087a1452f8cda8bb094b0642e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/b8584d43b27441c891d5dcea807d5e28, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/4976ce0c3ded4290950ffac896f7540e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/83d6453e8be74a65b7eab840b7986300, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/36a965be5831477c9acf89877e4c89b3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/1df6d40bab3f4200aad4410c1f53be26_SeqId_44_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp, totalSize=50.2 K 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 87f12ffc1f3f4ca3879d13886453931e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 5a5a58532f95448482a07c5f7ee337b6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 539392aeac254883a70057b9820ad8d3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 15d7c63277784856b34ef90986794771, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting d9ccc2a087a1452f8cda8bb094b0642e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting b8584d43b27441c891d5dcea807d5e28, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 4976ce0c3ded4290950ffac896f7540e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 83d6453e8be74a65b7eab840b7986300, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 36a965be5831477c9acf89877e4c89b3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-09 01:53:53,584 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 1df6d40bab3f4200aad4410c1f53be26_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-09 01:53:53,598 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:53,601 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:53:53,606 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228854007390dda4aaa9dba025242911547 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228854007390dda4aaa9dba025242911547 2023-03-09 01:53:53,608 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231ca3bdcd9b1954092a483ab573323474e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231ca3bdcd9b1954092a483ab573323474e 2023-03-09 01:53:53,610 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601017c627ce4b13e466095a572007f62c59a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601017c627ce4b13e466095a572007f62c59a 2023-03-09 01:53:53,612 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103358fb5e3d72546aba605c6a1b9721d71 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103358fb5e3d72546aba605c6a1b9721d71 2023-03-09 01:53:53,612 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-09 01:53:53,612 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-09 01:53:53,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:53,642 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 80524e4cfd7024a7ff50a654d546b0de#family1#compaction#10 average throughput is 0.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:53:54,076 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/60222098abd04dc186e30df8db1aa4d9 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/60222098abd04dc186e30df8db1aa4d9 2023-03-09 01:53:54,098 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 80524e4cfd7024a7ff50a654d546b0de/family1 of 80524e4cfd7024a7ff50a654d546b0de into 60222098abd04dc186e30df8db1aa4d9(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:53:54,098 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:54,100 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1678326833580; duration=0sec 2023-03-09 01:53:54,102 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:53:54,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/87f12ffc1f3f4ca3879d13886453931e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/5a5a58532f95448482a07c5f7ee337b6, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/539392aeac254883a70057b9820ad8d3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/15d7c63277784856b34ef90986794771, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/d9ccc2a087a1452f8cda8bb094b0642e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/b8584d43b27441c891d5dcea807d5e28, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/4976ce0c3ded4290950ffac896f7540e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/83d6453e8be74a65b7eab840b7986300, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/36a965be5831477c9acf89877e4c89b3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/1df6d40bab3f4200aad4410c1f53be26_SeqId_44_] to archive 2023-03-09 01:53:54,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:53:54,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/87f12ffc1f3f4ca3879d13886453931e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/87f12ffc1f3f4ca3879d13886453931e 2023-03-09 01:53:54,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/5a5a58532f95448482a07c5f7ee337b6 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/5a5a58532f95448482a07c5f7ee337b6 2023-03-09 01:53:54,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/539392aeac254883a70057b9820ad8d3 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/539392aeac254883a70057b9820ad8d3 2023-03-09 01:53:54,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/15d7c63277784856b34ef90986794771 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/15d7c63277784856b34ef90986794771 2023-03-09 01:53:54,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/d9ccc2a087a1452f8cda8bb094b0642e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/d9ccc2a087a1452f8cda8bb094b0642e 2023-03-09 01:53:54,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/b8584d43b27441c891d5dcea807d5e28 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/b8584d43b27441c891d5dcea807d5e28 2023-03-09 01:53:54,144 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/4976ce0c3ded4290950ffac896f7540e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/4976ce0c3ded4290950ffac896f7540e 2023-03-09 01:53:54,146 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/83d6453e8be74a65b7eab840b7986300 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/83d6453e8be74a65b7eab840b7986300 2023-03-09 01:53:54,148 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/36a965be5831477c9acf89877e4c89b3 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/36a965be5831477c9acf89877e4c89b3 2023-03-09 01:53:54,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/1df6d40bab3f4200aad4410c1f53be26_SeqId_44_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/1df6d40bab3f4200aad4410c1f53be26_SeqId_44_ 2023-03-09 01:53:54,507 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053927ba1b95914c93984fddff1b1bcef4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512053927ba1b95914c93984fddff1b1bcef4 2023-03-09 01:53:54,510 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:54,549 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/dbe852d63d084420b9de251225d24b36 first=Optional[r00] last=Optional[r2] 2023-03-09 01:53:54,553 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/dbe852d63d084420b9de251225d24b36}] 2023-03-09 01:53:54,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/dbe852d63d084420b9de251225d24b36 for inclusion in 80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:54,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-09 01:53:54,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:53:54,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:54,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/dbe852d63d084420b9de251225d24b36 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hvq8eq8kkrglq9adf9ta0jnrsnt5em0rnbg4l7ctho0cfp14c82ag8klovqjd293/family1/dbe852d63d084420b9de251225d24b36 2023-03-09 01:53:54,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hvq8eq8kkrglq9adf9ta0jnrsnt5em0rnbg4l7ctho0cfp14c82ag8klovqjd293/family1/dbe852d63d084420b9de251225d24b36 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_ 2023-03-09 01:53:54,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hvq8eq8kkrglq9adf9ta0jnrsnt5em0rnbg4l7ctho0cfp14c82ag8klovqjd293/family1/dbe852d63d084420b9de251225d24b36 into 80524e4cfd7024a7ff50a654d546b0de/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_ - updating store file list. 2023-03-09 01:53:54,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_ into 80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:54,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hvq8eq8kkrglq9adf9ta0jnrsnt5em0rnbg4l7ctho0cfp14c82ag8klovqjd293/family1/dbe852d63d084420b9de251225d24b36 into 80524e4cfd7024a7ff50a654d546b0de/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_) 2023-03-09 01:53:54,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__hvq8eq8kkrglq9adf9ta0jnrsnt5em0rnbg4l7ctho0cfp14c82ag8klovqjd293/family1/dbe852d63d084420b9de251225d24b36 2023-03-09 01:53:54,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-09 01:53:54,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-09 01:53:54,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 80524e4cfd7024a7ff50a654d546b0de/family1 is initiating minor compaction (all files) 2023-03-09 01:53:54,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@49492ae9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:53:54,577 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 80524e4cfd7024a7ff50a654d546b0de/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:54,577 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/fe1a1e8b364d4f44b2df6b459f509f5d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/60222098abd04dc186e30df8db1aa4d9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp, totalSize=358.6 K 2023-03-09 01:53:54,578 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting fe1a1e8b364d4f44b2df6b459f509f5d, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843359 2023-03-09 01:53:54,579 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 60222098abd04dc186e30df8db1aa4d9, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243359 2023-03-09 01:53:54,579 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878843359 2023-03-09 01:53:54,580 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:54,583 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:53:54,585 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130dba6cf58a39d40bfa2d93192d78b24c9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130dba6cf58a39d40bfa2d93192d78b24c9 2023-03-09 01:53:54,587 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512015fb8740e6ea445b981af318617117411 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512015fb8740e6ea445b981af318617117411 2023-03-09 01:53:54,589 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050fe29f704da647ec8a46d3ce565c0ff2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050fe29f704da647ec8a46d3ce565c0ff2 2023-03-09 01:53:54,589 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-09 01:53:54,589 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-09 01:53:54,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:54,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:54,591 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:53:54,591 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-09 01:53:54,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0670 sec 2023-03-09 01:53:54,623 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 80524e4cfd7024a7ff50a654d546b0de#family1#compaction#11 average throughput is 5.90 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:53:54,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-09 01:53:54,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:55,188 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-09 01:53:55,189 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-09 01:53:55,189 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-09 01:53:55,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-09 01:53:55,207 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=80524e4cfd7024a7ff50a654d546b0de, REOPEN/MOVE}] 2023-03-09 01:53:55,208 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=80524e4cfd7024a7ff50a654d546b0de, REOPEN/MOVE 2023-03-09 01:53:55,209 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=80524e4cfd7024a7ff50a654d546b0de, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:55,209 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678326835209"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326835209"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326835209"}]},"ts":"1678326835209"} 2023-03-09 01:53:55,212 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 80524e4cfd7024a7ff50a654d546b0de, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:53:55,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:55,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 80524e4cfd7024a7ff50a654d546b0de, disabling compactions & flushes 2023-03-09 01:53:55,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:55,852 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/.tmp/family1/d9518f5dab6044628242d520420c3d23 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/d9518f5dab6044628242d520420c3d23 2023-03-09 01:53:55,860 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 80524e4cfd7024a7ff50a654d546b0de/family1 of 80524e4cfd7024a7ff50a654d546b0de into d9518f5dab6044628242d520420c3d23(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:53:55,860 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:55,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:55,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:55,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. after waiting 0 ms 2023-03-09 01:53:55,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:55,860 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1678326834577; duration=1sec 2023-03-09 01:53:55,860 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:53:55,863 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/fe1a1e8b364d4f44b2df6b459f509f5d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/60222098abd04dc186e30df8db1aa4d9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_] to archive 2023-03-09 01:53:55,864 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:53:55,866 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/fe1a1e8b364d4f44b2df6b459f509f5d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/fe1a1e8b364d4f44b2df6b459f509f5d 2023-03-09 01:53:55,868 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/60222098abd04dc186e30df8db1aa4d9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/60222098abd04dc186e30df8db1aa4d9 2023-03-09 01:53:55,869 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/de485d7d44e74efa9503b05e8eb7a59d_SeqId_47_ 2023-03-09 01:53:55,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-09 01:53:55,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:55,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:55,877 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 80524e4cfd7024a7ff50a654d546b0de to self. 2023-03-09 01:53:55,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:55,880 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=80524e4cfd7024a7ff50a654d546b0de, regionState=CLOSED 2023-03-09 01:53:55,880 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678326835880"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326835880"}]},"ts":"1678326835880"} 2023-03-09 01:53:55,885 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-09 01:53:55,885 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 80524e4cfd7024a7ff50a654d546b0de, server=jenkins-hbase4.apache.org,46511,1678326819757 in 670 msec 2023-03-09 01:53:55,886 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=80524e4cfd7024a7ff50a654d546b0de, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=true 2023-03-09 01:53:56,036 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=80524e4cfd7024a7ff50a654d546b0de, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:56,037 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678326836036"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326836036"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326836036"}]},"ts":"1678326836036"} 2023-03-09 01:53:56,039 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 80524e4cfd7024a7ff50a654d546b0de, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:53:56,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:56,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 80524e4cfd7024a7ff50a654d546b0de, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:53:56,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:56,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:56,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:56,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:56,198 INFO [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:56,200 DEBUG [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:56,200 DEBUG [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:56,200 INFO [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80524e4cfd7024a7ff50a654d546b0de columnFamilyName family1 2023-03-09 01:53:56,207 DEBUG [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/d9518f5dab6044628242d520420c3d23 2023-03-09 01:53:56,208 INFO [StoreOpener-80524e4cfd7024a7ff50a654d546b0de-1] regionserver.HStore(310): Store=80524e4cfd7024a7ff50a654d546b0de/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:56,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:56,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:56,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:53:56,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 80524e4cfd7024a7ff50a654d546b0de; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64300948, jitterRate=-0.041841208934783936}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:53:56,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:56,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de., pid=17, masterSystemTime=1678326836192 2023-03-09 01:53:56,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:56,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:53:56,219 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=80524e4cfd7024a7ff50a654d546b0de, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:56,219 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1678326836219"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326836219"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326836219"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326836219"}]},"ts":"1678326836219"} 2023-03-09 01:53:56,223 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-09 01:53:56,223 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 80524e4cfd7024a7ff50a654d546b0de, server=jenkins-hbase4.apache.org,46511,1678326819757 in 182 msec 2023-03-09 01:53:56,225 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-09 01:53:56,225 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=80524e4cfd7024a7ff50a654d546b0de, REOPEN/MOVE in 1.0170 sec 2023-03-09 01:53:56,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-09 01:53:56,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0330 sec 2023-03-09 01:53:56,230 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4670 sec 2023-03-09 01:53:56,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-09 01:53:56,240 INFO [Listener at localhost/33973] compactions.TestMobCompactor(278): alter status finished 2023-03-09 01:53:56,241 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-09 01:53:56,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-09 01:53:56,243 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-09 01:53:56,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:53:56,244 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327436244 2023-03-09 01:53:56,245 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:53:56,246 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-09 01:53:56,246 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:56,247 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x6aa3b364 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:53:56,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1fb3444a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:53:56,252 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:56,252 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:56,252 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:56,252 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:56,252 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:57,087 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128bab118fd8f5b43478476ed877c461ad4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128bab118fd8f5b43478476ed877c461ad4 2023-03-09 01:53:57,088 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:57,107 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:53:57,109 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52918, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:53:57,116 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:53:57,118 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44374, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:53:57,129 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/67aad311c8ec4b7e8a6e585fc5553d61 first=Optional[r05] last=Optional[r8] 2023-03-09 01:53:57,130 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/67aad311c8ec4b7e8a6e585fc5553d61}] 2023-03-09 01:53:57,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/67aad311c8ec4b7e8a6e585fc5553d61 for inclusion in 80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:57,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-09 01:53:57,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:53:57,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:53:57,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/67aad311c8ec4b7e8a6e585fc5553d61 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ckrejqhu0bi0s5pc8heo21gfr4iuilv6nt6r67bg9c3c2s4nv2s4kbff1bdrtlav/family1/67aad311c8ec4b7e8a6e585fc5553d61 2023-03-09 01:53:57,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ckrejqhu0bi0s5pc8heo21gfr4iuilv6nt6r67bg9c3c2s4nv2s4kbff1bdrtlav/family1/67aad311c8ec4b7e8a6e585fc5553d61 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/67fe9830d14e45baa0131fa5b726749e_SeqId_53_ 2023-03-09 01:53:57,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ckrejqhu0bi0s5pc8heo21gfr4iuilv6nt6r67bg9c3c2s4nv2s4kbff1bdrtlav/family1/67aad311c8ec4b7e8a6e585fc5553d61 into 80524e4cfd7024a7ff50a654d546b0de/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/67fe9830d14e45baa0131fa5b726749e_SeqId_53_ - updating store file list. 2023-03-09 01:53:57,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/67fe9830d14e45baa0131fa5b726749e_SeqId_53_ into 80524e4cfd7024a7ff50a654d546b0de/family1 2023-03-09 01:53:57,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ckrejqhu0bi0s5pc8heo21gfr4iuilv6nt6r67bg9c3c2s4nv2s4kbff1bdrtlav/family1/67aad311c8ec4b7e8a6e585fc5553d61 into 80524e4cfd7024a7ff50a654d546b0de/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/family1/67fe9830d14e45baa0131fa5b726749e_SeqId_53_) 2023-03-09 01:53:57,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ckrejqhu0bi0s5pc8heo21gfr4iuilv6nt6r67bg9c3c2s4nv2s4kbff1bdrtlav/family1/67aad311c8ec4b7e8a6e585fc5553d61 2023-03-09 01:53:57,153 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:57,154 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:53:57,155 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120293b261419274fdf8a6cb684dd99f320 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120293b261419274fdf8a6cb684dd99f320 2023-03-09 01:53:57,157 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511286e08558ba1144c9d9ccee1f0041e0b7e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511286e08558ba1144c9d9ccee1f0041e0b7e 2023-03-09 01:53:57,157 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-09 01:53:57,157 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-09 01:53:57,157 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:57,158 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:57,158 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-09 01:53:57,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-09 01:53:57,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:53:57,159 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-09 01:53:57,161 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 917 msec 2023-03-09 01:53:57,239 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=247 (was 211) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:53550 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:53574 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x6aa3b364-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:53530 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: (jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40834 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40738 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40790 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4ae69a45-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x41e2de52-SendThread(127.0.0.1:50548) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79dabcc6-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79dabcc6-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79dabcc6-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40716 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:53602 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x6aa3b364-SendThread(127.0.0.1:50548) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:53564 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x41e2de52 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/482268361.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40724 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x6aa3b364 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/482268361.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40772 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40756 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40838 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x41e2de52-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4ae69a45-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:46511-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-984539750_15 at /127.0.0.1:53590 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40722 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/cluster_c7ae5eb2-da64-f932-cf16-ebb509eb1321/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=591 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=289 (was 353), ProcessCount=173 (was 173), AvailableMemoryMB=8071 (was 8355) 2023-03-09 01:53:57,252 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=247, OpenFileDescriptor=591, MaxFileDescriptor=60000, SystemLoadAverage=289, ProcessCount=173, AvailableMemoryMB=8070 2023-03-09 01:53:57,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-09 01:53:57,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:57,257 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:53:57,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-09 01:53:57,258 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:53:57,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-09 01:53:57,260 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:57,260 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48 empty. 2023-03-09 01:53:57,261 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:57,261 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-09 01:53:57,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-09 01:53:57,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-09 01:53:57,671 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-09 01:53:57,675 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 519ef0416144e355c478496bd9b26e48, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:53:57,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:57,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 519ef0416144e355c478496bd9b26e48, disabling compactions & flushes 2023-03-09 01:53:57,699 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:57,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:57,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. after waiting 0 ms 2023-03-09 01:53:57,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:57,699 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:57,699 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:53:57,702 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:53:57,703 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678326837703"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326837703"}]},"ts":"1678326837703"} 2023-03-09 01:53:57,706 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:53:57,707 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:53:57,707 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326837707"}]},"ts":"1678326837707"} 2023-03-09 01:53:57,709 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-09 01:53:57,714 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=519ef0416144e355c478496bd9b26e48, ASSIGN}] 2023-03-09 01:53:57,716 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=519ef0416144e355c478496bd9b26e48, ASSIGN 2023-03-09 01:53:57,717 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=519ef0416144e355c478496bd9b26e48, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:53:57,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-09 01:53:57,868 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=519ef0416144e355c478496bd9b26e48, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:57,868 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678326837868"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326837868"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326837868"}]},"ts":"1678326837868"} 2023-03-09 01:53:57,870 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 519ef0416144e355c478496bd9b26e48, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:53:58,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:58,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 519ef0416144e355c478496bd9b26e48, NAME => 'testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:53:58,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:58,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:53:58,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:58,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:58,028 INFO [StoreOpener-519ef0416144e355c478496bd9b26e48-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:58,029 DEBUG [StoreOpener-519ef0416144e355c478496bd9b26e48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1 2023-03-09 01:53:58,029 DEBUG [StoreOpener-519ef0416144e355c478496bd9b26e48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1 2023-03-09 01:53:58,030 INFO [StoreOpener-519ef0416144e355c478496bd9b26e48-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 519ef0416144e355c478496bd9b26e48 columnFamilyName family1 2023-03-09 01:53:58,032 INFO [StoreOpener-519ef0416144e355c478496bd9b26e48-1] regionserver.HStore(310): Store=519ef0416144e355c478496bd9b26e48/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:53:58,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:58,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:58,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 519ef0416144e355c478496bd9b26e48 2023-03-09 01:53:58,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:53:58,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 519ef0416144e355c478496bd9b26e48; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75144468, jitterRate=0.119739830493927}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:53:58,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:53:58,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48., pid=21, masterSystemTime=1678326838022 2023-03-09 01:53:58,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:58,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:58,043 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=519ef0416144e355c478496bd9b26e48, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,043 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678326838043"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326838043"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326838043"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326838043"}]},"ts":"1678326838043"} 2023-03-09 01:53:58,047 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-09 01:53:58,047 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 519ef0416144e355c478496bd9b26e48, server=jenkins-hbase4.apache.org,46511,1678326819757 in 175 msec 2023-03-09 01:53:58,050 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-09 01:53:58,050 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=519ef0416144e355c478496bd9b26e48, ASSIGN in 333 msec 2023-03-09 01:53:58,050 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:53:58,051 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326838051"}]},"ts":"1678326838051"} 2023-03-09 01:53:58,052 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-09 01:53:58,055 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:53:58,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 800 msec 2023-03-09 01:53:58,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-09 01:53:58,364 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-09 01:53:58,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:58,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,408 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:53:58,408 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:58,409 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:53:58,409 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:53:58,409 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,410 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,411 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:58,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:58,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:58,411 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,412 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,412 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:58,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:58,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,413 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,413 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:53:58,413 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:58,415 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:53:58,416 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:58,416 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:58,416 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:58,416 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:53:58,417 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-09 01:53:58,459 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113025fab766ab19498f83d6cc8926821ce4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113025fab766ab19498f83d6cc8926821ce4 2023-03-09 01:53:58,464 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/58e82328cb1c4b7884a095a235046224 2023-03-09 01:53:58,472 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/58e82328cb1c4b7884a095a235046224 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/58e82328cb1c4b7884a095a235046224 2023-03-09 01:53:58,481 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/58e82328cb1c4b7884a095a235046224, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-09 01:53:58,483 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 67ms, sequenceid=5, compaction requested=false 2023-03-09 01:53:58,483 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:53:58,484 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:53:58,484 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,485 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:53:58,485 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:53:58,487 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,487 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:58,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:58,488 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,488 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:58,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:58,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:58,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:58,490 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:53:58,490 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@674a3074[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:58,490 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:53:58,490 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:58,492 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,492 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:53:58,492 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:53:58,492 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:58,495 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,495 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:58,495 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:58,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:53:58,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:58,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:58,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:58,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:58,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:53:58,501 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:53:58,501 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:58,501 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:58,501 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,501 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:58,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:58,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:58,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:58,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:58,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:58,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:58,506 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:58,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:58,509 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:58,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:58,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:58,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:53:58,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:58,512 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:58,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:58,513 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:58,514 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:58,614 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:58,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:58,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:58,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:58,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,632 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:53:58,632 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:58,633 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:53:58,633 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:53:58,633 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,633 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,636 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,636 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:58,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:58,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:58,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,637 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:58,637 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:58,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:58,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,640 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:53:58,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:58,640 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:53:58,640 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:58,641 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:53:58,644 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:58,644 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:58,644 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:58,644 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:53:58,644 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-09 01:53:59,073 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511286ffd5655757e49abbaee90a909b856c7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511286ffd5655757e49abbaee90a909b856c7 2023-03-09 01:53:59,479 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/85c26ddfad0f46cdb38921c5b1644686 2023-03-09 01:53:59,488 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/85c26ddfad0f46cdb38921c5b1644686 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/85c26ddfad0f46cdb38921c5b1644686 2023-03-09 01:53:59,497 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/85c26ddfad0f46cdb38921c5b1644686, entries=1, sequenceid=9, filesize=5.0 K 2023-03-09 01:53:59,498 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 854ms, sequenceid=9, compaction requested=false 2023-03-09 01:53:59,499 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:53:59,499 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:59,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:53:59,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:53:59,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:53:59,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:53:59,501 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,501 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:59,502 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,502 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:53:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:59,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:59,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:53:59,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34e2a227[Count = 0] remaining members to acquire global barrier 2023-03-09 01:53:59,504 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:53:59,504 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,510 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,510 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:53:59,510 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:53:59,510 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:53:59,510 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,510 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:53:59,511 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,511 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:53:59,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:59,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:59,512 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:59,512 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:53:59,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:59,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:59,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:53:59,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:53:59,515 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:53:59,515 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:53:59,515 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:53:59,515 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,515 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:53:59,517 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,517 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:59,517 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:53:59,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:53:59,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:59,518 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:59,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:53:59,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:53:59,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,520 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:53:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:59,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:59,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:53:59,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:53:59,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:59,526 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:53:59,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:53:59,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,527 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,527 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,527 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:53:59,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:53:59,527 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,527 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:53:59,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:53:59,528 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:53:59,628 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:53:59,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:53:59,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:53:59,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:53:59,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,635 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:53:59,635 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:59,636 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:53:59,636 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:53:59,636 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,636 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,638 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:53:59,638 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:53:59,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:53:59,638 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:53:59,638 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:53:59,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:53:59,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,639 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,639 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:53:59,639 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:53:59,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:53:59,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:53:59,640 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:53:59,640 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:53:59,640 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:53:59,640 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:53:59,640 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:53:59,640 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:00,055 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512018d4910330e3445c3962ee329d2d99001 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512018d4910330e3445c3962ee329d2d99001 2023-03-09 01:54:00,462 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/191f0991666d4e808fd06e11154fb499 2023-03-09 01:54:00,470 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/191f0991666d4e808fd06e11154fb499 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/191f0991666d4e808fd06e11154fb499 2023-03-09 01:54:00,475 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/191f0991666d4e808fd06e11154fb499, entries=1, sequenceid=13, filesize=4.9 K 2023-03-09 01:54:00,477 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 837ms, sequenceid=13, compaction requested=true 2023-03-09 01:54:00,477 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:00,477 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:54:00,477 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:54:00,538 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,538 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:00,538 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:00,538 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:00,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:00,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:54:00,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4106620c[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:00,540 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:00,540 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,576 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,576 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:00,576 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:54:00,576 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:00,576 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,576 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:00,607 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:54:00,607 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,607 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:00,607 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:54:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:00,610 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:00,610 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:54:00,610 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:00,610 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:00,610 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,610 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:00,614 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,614 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:00,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:00,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,614 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:00,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,615 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:00,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:00,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:00,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:00,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,616 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,622 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,622 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:00,622 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:00,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:00,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:00,622 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:54:00,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:00,622 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:00,622 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:00,623 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:00,622 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,623 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:00,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:00,623 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:00,623 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,624 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,624 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,723 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:00,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:00,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:00,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:00,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,732 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:00,732 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:00,733 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:54:00,733 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:00,734 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,734 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,736 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,736 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:00,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:00,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:00,737 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:00,737 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:00,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:00,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,738 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:54:00,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:00,738 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:54:00,738 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:00,739 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:54:00,739 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:00,739 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:00,739 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:00,739 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:54:00,739 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:01,156 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205790ef46b2b78410eb79c0a7ced1e404d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205790ef46b2b78410eb79c0a7ced1e404d 2023-03-09 01:54:01,411 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:54:01,562 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/187b09cd1cc146639e31d6aa0a8dd2f6 2023-03-09 01:54:01,568 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/187b09cd1cc146639e31d6aa0a8dd2f6 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/187b09cd1cc146639e31d6aa0a8dd2f6 2023-03-09 01:54:01,573 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/187b09cd1cc146639e31d6aa0a8dd2f6, entries=1, sequenceid=17, filesize=4.9 K 2023-03-09 01:54:01,574 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 835ms, sequenceid=17, compaction requested=true 2023-03-09 01:54:01,574 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:01,574 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:01,574 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:01,574 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:01,574 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,574 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:54:01,574 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:54:01,577 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,577 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:01,577 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,577 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:01,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:01,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:01,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:54:01,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59ee3b3f[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:01,579 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:01,579 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,580 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,580 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:01,580 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:54:01,580 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,580 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:01,580 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:01,582 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:54:01,582 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:01,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,582 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:54:01,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:01,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:01,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:01,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:01,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:01,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:01,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:01,584 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:54:01,585 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:01,585 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:01,585 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,585 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:01,586 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,586 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,586 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:01,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:01,587 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:01,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,588 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:01,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,591 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,591 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,591 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:01,591 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:01,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:01,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:01,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:01,591 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:01,591 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,591 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:01,592 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,592 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,592 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,592 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:01,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:01,593 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:01,693 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:01,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:01,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:01,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:01,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,702 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:01,702 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:01,702 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:54:01,702 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:01,703 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,703 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,705 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,705 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:01,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:01,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:01,705 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:01,705 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:01,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,706 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:54:01,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:01,707 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:54:01,707 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:01,707 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:54:01,707 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:01,707 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:01,707 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:01,708 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:54:01,708 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:01,725 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283bc5d2cbc6d94e5cb106aa537aed1097 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512283bc5d2cbc6d94e5cb106aa537aed1097 2023-03-09 01:54:02,132 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/7585827f7c8c4693a5ab5acaab1d0a13 2023-03-09 01:54:02,138 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/7585827f7c8c4693a5ab5acaab1d0a13 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7585827f7c8c4693a5ab5acaab1d0a13 2023-03-09 01:54:02,144 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7585827f7c8c4693a5ab5acaab1d0a13, entries=1, sequenceid=21, filesize=4.9 K 2023-03-09 01:54:02,146 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 438ms, sequenceid=21, compaction requested=true 2023-03-09 01:54:02,146 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:02,146 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:02,146 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:02,146 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:02,146 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,146 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:54:02,146 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:54:02,148 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,148 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:02,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:02,149 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,149 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:02,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:54:02,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f9179e6[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:02,150 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:02,150 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,151 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,151 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:02,152 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:54:02,152 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:02,152 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,152 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:02,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,153 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:54:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:02,153 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:02,153 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:54:02,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:02,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:02,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:02,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:02,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:02,157 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:54:02,157 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:02,157 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:02,157 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,157 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:02,165 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,165 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:02,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:02,165 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:02,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,166 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:02,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:02,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:02,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,171 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,171 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:02,171 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,171 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:02,171 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:02,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:02,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:02,171 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:02,172 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,172 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,172 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,172 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,172 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:02,172 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:02,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:02,272 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:02,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:02,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:02,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:02,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,280 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:02,280 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:02,280 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:54:02,280 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:02,281 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,281 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,282 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,282 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:02,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:02,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:02,283 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:02,283 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:02,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,283 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:02,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,283 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,284 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:54:02,284 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:02,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:54:02,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:02,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:54:02,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:02,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:02,285 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:02,285 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:54:02,285 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:02,699 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512312e1917c5be2147a0b5df4ac91cfb9f36 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312e1917c5be2147a0b5df4ac91cfb9f36 2023-03-09 01:54:03,106 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/b9045dce73224be6b6128cd2195aa5e8 2023-03-09 01:54:03,114 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/b9045dce73224be6b6128cd2195aa5e8 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/b9045dce73224be6b6128cd2195aa5e8 2023-03-09 01:54:03,120 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/b9045dce73224be6b6128cd2195aa5e8, entries=1, sequenceid=25, filesize=4.9 K 2023-03-09 01:54:03,121 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 836ms, sequenceid=25, compaction requested=true 2023-03-09 01:54:03,122 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:03,122 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:03,122 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:03,122 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:03,122 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,122 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:54:03,122 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:54:03,124 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,124 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:03,124 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,124 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:03,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:03,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:54:03,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ec31da9[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:03,126 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:03,126 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,128 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,129 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:03,129 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:54:03,129 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:03,129 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,129 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:03,132 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:54:03,132 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,132 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:03,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:03,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:03,132 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:54:03,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:03,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:03,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:03,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:03,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:03,135 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:54:03,135 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:03,135 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:03,135 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,135 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:03,138 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,138 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,138 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:03,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:03,139 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:03,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,140 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,143 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,143 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,143 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:03,143 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:03,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:03,143 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:03,143 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:03,143 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:03,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:54:03,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:03,144 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,144 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:03,145 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:03,145 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,145 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:03,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:03,245 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:03,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:03,251 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:03,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:03,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,253 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:03,253 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:03,254 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:54:03,254 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:03,254 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,255 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,257 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,257 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:03,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:03,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:03,257 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:03,257 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:03,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:03,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,258 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,258 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:54:03,258 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:03,258 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:54:03,258 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:03,259 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:54:03,259 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:03,259 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:03,259 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:03,259 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:54:03,259 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:03,680 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010142b5abb1143c482f98c41dae3d41217c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010142b5abb1143c482f98c41dae3d41217c 2023-03-09 01:54:04,087 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/2026a0d5c99b4a8cbc855b66d962ebd2 2023-03-09 01:54:04,095 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/2026a0d5c99b4a8cbc855b66d962ebd2 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/2026a0d5c99b4a8cbc855b66d962ebd2 2023-03-09 01:54:04,102 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/2026a0d5c99b4a8cbc855b66d962ebd2, entries=1, sequenceid=29, filesize=4.9 K 2023-03-09 01:54:04,103 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 844ms, sequenceid=29, compaction requested=true 2023-03-09 01:54:04,103 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:04,103 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:04,103 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:04,103 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:04,103 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,104 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:54:04,104 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:54:04,105 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:04,106 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,106 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:54:04,107 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ca7107f[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:04,107 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:04,107 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,109 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,109 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:04,109 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:54:04,109 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:04,109 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,109 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:04,110 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:54:04,111 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:04,111 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,111 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:54:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:04,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:04,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:04,114 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:54:04,114 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:04,114 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:04,114 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,114 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:04,117 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,117 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:04,117 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,117 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:04,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:04,118 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,121 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,121 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:04,121 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,121 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:04,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:04,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:04,121 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:04,121 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:04,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:54:04,122 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,122 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:04,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:04,122 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:04,122 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:04,122 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,123 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,222 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:04,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:04,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:04,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:04,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,232 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:04,232 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:04,232 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:54:04,232 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:04,233 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,233 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,234 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,234 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:04,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:04,235 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,235 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:04,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:04,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:54:04,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,236 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:54:04,236 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:04,237 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:54:04,237 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:04,237 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:04,237 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:04,237 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:54:04,237 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:04,656 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038dae37ba06ff41f198f20ab41a7244f2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601038dae37ba06ff41f198f20ab41a7244f2 2023-03-09 01:54:04,663 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/0468dd7668f347a5b77159327f5fbb77 2023-03-09 01:54:04,670 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/0468dd7668f347a5b77159327f5fbb77 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/0468dd7668f347a5b77159327f5fbb77 2023-03-09 01:54:04,676 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/0468dd7668f347a5b77159327f5fbb77, entries=1, sequenceid=33, filesize=4.9 K 2023-03-09 01:54:04,677 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 440ms, sequenceid=33, compaction requested=true 2023-03-09 01:54:04,677 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:04,677 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:04,678 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:04,678 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:04,678 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,678 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:54:04,678 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:54:04,680 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:04,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:04,680 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,680 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,680 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:04,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:04,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:04,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:04,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:54:04,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1438f317[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:04,682 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:04,683 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,685 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,685 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:04,685 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,685 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:04,685 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:54:04,685 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:04,692 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:54:04,692 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,693 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:04,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,693 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:54:04,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:04,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:04,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:04,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:04,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:04,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:04,696 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:04,696 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:54:04,696 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:04,696 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:04,696 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,696 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:04,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:04,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:04,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:04,699 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:04,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:04,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:04,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:04,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,700 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:04,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:04,707 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:54:04,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:04,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:04,707 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:04,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:04,708 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:04,708 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:04,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:04,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:04,809 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:04,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:04,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:04,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:04,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,818 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:04,819 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:04,819 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:54:04,819 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:04,819 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,819 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,821 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,821 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:04,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:04,821 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:04,821 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,822 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:04,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,822 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:54:04,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:04,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:54:04,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:04,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:54:04,824 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:04,824 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:04,824 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:04,824 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:54:04,824 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:05,244 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509071ef37d66f0774a44aded3f6f94cc972b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509071ef37d66f0774a44aded3f6f94cc972b 2023-03-09 01:54:05,254 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/3b83ff0130a045299864452ef3441f62 2023-03-09 01:54:05,269 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/3b83ff0130a045299864452ef3441f62 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/3b83ff0130a045299864452ef3441f62 2023-03-09 01:54:05,276 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/3b83ff0130a045299864452ef3441f62, entries=1, sequenceid=37, filesize=4.9 K 2023-03-09 01:54:05,278 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 454ms, sequenceid=37, compaction requested=true 2023-03-09 01:54:05,278 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:05,278 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:54:05,278 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:54:05,280 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,280 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:05,281 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,281 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:05,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:54:05,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7082e63f[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:05,282 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:05,283 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:05,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:54:05,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:05,285 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,285 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:05,287 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:54:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:05,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:05,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:05,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:05,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:05,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:05,291 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:54:05,291 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:05,291 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:05,291 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,291 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:05,293 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,294 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,294 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:05,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:05,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,295 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,297 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,300 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:05,300 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:05,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:05,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:05,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:05,300 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,305 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,305 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,305 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,305 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:05,300 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:05,305 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,305 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:05,306 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:05,306 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:05,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:05,406 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:05,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:05,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:05,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:05,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,414 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:05,414 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:05,415 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:54:05,415 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:05,415 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,415 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,417 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:05,417 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:05,417 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,417 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,418 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,418 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:54:05,418 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:05,420 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:54:05,420 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:05,420 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:05,420 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:05,420 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. started... 2023-03-09 01:54:05,420 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 519ef0416144e355c478496bd9b26e48 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:05,439 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120c031900904d44eeca15ba53221b27d1a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120c031900904d44eeca15ba53221b27d1a 2023-03-09 01:54:05,446 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/7cc79fb9e68641c28fbbad17fe83e667 2023-03-09 01:54:05,455 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/7cc79fb9e68641c28fbbad17fe83e667 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7cc79fb9e68641c28fbbad17fe83e667 2023-03-09 01:54:05,462 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7cc79fb9e68641c28fbbad17fe83e667, entries=1, sequenceid=41, filesize=4.9 K 2023-03-09 01:54:05,463 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 519ef0416144e355c478496bd9b26e48 in 43ms, sequenceid=41, compaction requested=true 2023-03-09 01:54:05,464 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:05,464 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:05,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:05,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:05,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:54:05,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:54:05,466 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,466 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:05,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:05,468 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,468 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:05,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:05,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:05,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:05,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:54:05,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@267a83d3[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:05,470 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:05,470 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,471 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,471 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:05,472 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:54:05,472 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:05,472 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,472 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:05,473 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,473 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:05,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:05,473 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:05,473 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:05,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:05,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:05,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:05,476 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:54:05,476 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:05,476 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:05,476 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,476 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:05,478 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,478 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:05,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,478 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:05,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:05,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:05,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:05,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:05,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:05,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:05,481 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,482 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:05,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,486 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:05,486 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:05,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-09 01:54:05,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:05,486 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:05,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:05,486 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:05,487 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,487 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,487 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:05,488 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,488 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:05,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:05,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:05,588 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:05,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:05,591 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-09 01:54:05,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:05,593 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:05,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:05,594 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327445594 2023-03-09 01:54:05,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:54:05,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-09 01:54:05,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:54:05,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x53e7b61f to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:54:05,602 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5673393d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:54:05,602 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:54:06,444 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103cd781ec0197d44a59a2513eed1822553 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103cd781ec0197d44a59a2513eed1822553 2023-03-09 01:54:06,445 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:54:06,458 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:54:06,459 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52724, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:54:06,466 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:54:06,467 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52000, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:54:06,477 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ff5b55188a494716b4b53fa3ceb78185 first=Optional[r5] last=Optional[r6] 2023-03-09 01:54:06,478 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ff5b55188a494716b4b53fa3ceb78185}] 2023-03-09 01:54:06,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ff5b55188a494716b4b53fa3ceb78185 for inclusion in 519ef0416144e355c478496bd9b26e48/family1 2023-03-09 01:54:06,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-09 01:54:06,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:06,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:06,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ff5b55188a494716b4b53fa3ceb78185 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qufgpq1msmdi01f7213i27odfpb6vmgd6kd3li4d80vs5lf1rmoueevrah4ihl8/family1/ff5b55188a494716b4b53fa3ceb78185 2023-03-09 01:54:06,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qufgpq1msmdi01f7213i27odfpb6vmgd6kd3li4d80vs5lf1rmoueevrah4ihl8/family1/ff5b55188a494716b4b53fa3ceb78185 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/c2e8488db66043bb8fb27d5d0861797a_SeqId_44_ 2023-03-09 01:54:06,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qufgpq1msmdi01f7213i27odfpb6vmgd6kd3li4d80vs5lf1rmoueevrah4ihl8/family1/ff5b55188a494716b4b53fa3ceb78185 into 519ef0416144e355c478496bd9b26e48/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/c2e8488db66043bb8fb27d5d0861797a_SeqId_44_ - updating store file list. 2023-03-09 01:54:06,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/c2e8488db66043bb8fb27d5d0861797a_SeqId_44_ into 519ef0416144e355c478496bd9b26e48/family1 2023-03-09 01:54:06,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qufgpq1msmdi01f7213i27odfpb6vmgd6kd3li4d80vs5lf1rmoueevrah4ihl8/family1/ff5b55188a494716b4b53fa3ceb78185 into 519ef0416144e355c478496bd9b26e48/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/c2e8488db66043bb8fb27d5d0861797a_SeqId_44_) 2023-03-09 01:54:06,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qufgpq1msmdi01f7213i27odfpb6vmgd6kd3li4d80vs5lf1rmoueevrah4ihl8/family1/ff5b55188a494716b4b53fa3ceb78185 2023-03-09 01:54:06,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-09 01:54:06,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-09 01:54:06,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 519ef0416144e355c478496bd9b26e48/family1 is initiating minor compaction 2023-03-09 01:54:06,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@569e5644; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:06,498 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 519ef0416144e355c478496bd9b26e48/family1 in testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:06,499 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/85c26ddfad0f46cdb38921c5b1644686, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/191f0991666d4e808fd06e11154fb499, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/187b09cd1cc146639e31d6aa0a8dd2f6, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7585827f7c8c4693a5ab5acaab1d0a13, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/b9045dce73224be6b6128cd2195aa5e8, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/2026a0d5c99b4a8cbc855b66d962ebd2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/0468dd7668f347a5b77159327f5fbb77, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/3b83ff0130a045299864452ef3441f62, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7cc79fb9e68641c28fbbad17fe83e667, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/c2e8488db66043bb8fb27d5d0861797a_SeqId_44_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp, totalSize=49.7 K 2023-03-09 01:54:06,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 85c26ddfad0f46cdb38921c5b1644686, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-09 01:54:06,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 191f0991666d4e808fd06e11154fb499, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-09 01:54:06,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 187b09cd1cc146639e31d6aa0a8dd2f6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-09 01:54:06,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 7585827f7c8c4693a5ab5acaab1d0a13, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-09 01:54:06,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting b9045dce73224be6b6128cd2195aa5e8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-09 01:54:06,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 2026a0d5c99b4a8cbc855b66d962ebd2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-09 01:54:06,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 0468dd7668f347a5b77159327f5fbb77, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-09 01:54:06,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 3b83ff0130a045299864452ef3441f62, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-09 01:54:06,500 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 7cc79fb9e68641c28fbbad17fe83e667, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-09 01:54:06,500 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting c2e8488db66043bb8fb27d5d0861797a_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-09 01:54:06,500 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:54:06,501 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:06,503 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010142b5abb1143c482f98c41dae3d41217c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010142b5abb1143c482f98c41dae3d41217c 2023-03-09 01:54:06,505 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601038dae37ba06ff41f198f20ab41a7244f2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601038dae37ba06ff41f198f20ab41a7244f2 2023-03-09 01:54:06,505 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-09 01:54:06,505 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-09 01:54:06,506 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:54:06,506 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:54:06,542 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 519ef0416144e355c478496bd9b26e48#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:54:06,561 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316e0c65bdeaf04a61ac11e904208e900d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512316e0c65bdeaf04a61ac11e904208e900d 2023-03-09 01:54:06,562 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:54:06,588 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/98ff13143922491081e30ed190fd2562 first=Optional[r1] last=Optional[r4] 2023-03-09 01:54:06,589 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/98ff13143922491081e30ed190fd2562}] 2023-03-09 01:54:06,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/98ff13143922491081e30ed190fd2562 for inclusion in 519ef0416144e355c478496bd9b26e48/family1 2023-03-09 01:54:06,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-09 01:54:06,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:06,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:06,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/98ff13143922491081e30ed190fd2562 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__nu5e1alk621o81i0em5l5qe6qvm2slguoblt6d1iv2lcvni0gqdgplalndhe3ccm/family1/98ff13143922491081e30ed190fd2562 2023-03-09 01:54:06,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__nu5e1alk621o81i0em5l5qe6qvm2slguoblt6d1iv2lcvni0gqdgplalndhe3ccm/family1/98ff13143922491081e30ed190fd2562 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_ 2023-03-09 01:54:06,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__nu5e1alk621o81i0em5l5qe6qvm2slguoblt6d1iv2lcvni0gqdgplalndhe3ccm/family1/98ff13143922491081e30ed190fd2562 into 519ef0416144e355c478496bd9b26e48/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_ - updating store file list. 2023-03-09 01:54:06,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_ into 519ef0416144e355c478496bd9b26e48/family1 2023-03-09 01:54:06,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__nu5e1alk621o81i0em5l5qe6qvm2slguoblt6d1iv2lcvni0gqdgplalndhe3ccm/family1/98ff13143922491081e30ed190fd2562 into 519ef0416144e355c478496bd9b26e48/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_) 2023-03-09 01:54:06,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__nu5e1alk621o81i0em5l5qe6qvm2slguoblt6d1iv2lcvni0gqdgplalndhe3ccm/family1/98ff13143922491081e30ed190fd2562 2023-03-09 01:54:06,605 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:54:06,606 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:06,607 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512018d4910330e3445c3962ee329d2d99001 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512018d4910330e3445c3962ee329d2d99001 2023-03-09 01:54:06,608 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205790ef46b2b78410eb79c0a7ced1e404d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205790ef46b2b78410eb79c0a7ced1e404d 2023-03-09 01:54:06,609 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512283bc5d2cbc6d94e5cb106aa537aed1097 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512283bc5d2cbc6d94e5cb106aa537aed1097 2023-03-09 01:54:06,611 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312e1917c5be2147a0b5df4ac91cfb9f36 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312e1917c5be2147a0b5df4ac91cfb9f36 2023-03-09 01:54:06,611 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-09 01:54:06,611 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-09 01:54:06,611 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:54:06,611 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:54:06,961 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/27d2c85f4151499a856c9daff44adf0f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/27d2c85f4151499a856c9daff44adf0f 2023-03-09 01:54:06,968 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 519ef0416144e355c478496bd9b26e48/family1 of 519ef0416144e355c478496bd9b26e48 into 27d2c85f4151499a856c9daff44adf0f(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:54:06,968 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:06,968 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678326846498; duration=0sec 2023-03-09 01:54:06,968 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:07,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/85c26ddfad0f46cdb38921c5b1644686, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/191f0991666d4e808fd06e11154fb499, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/187b09cd1cc146639e31d6aa0a8dd2f6, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7585827f7c8c4693a5ab5acaab1d0a13, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/b9045dce73224be6b6128cd2195aa5e8, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/2026a0d5c99b4a8cbc855b66d962ebd2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/0468dd7668f347a5b77159327f5fbb77, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/3b83ff0130a045299864452ef3441f62, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7cc79fb9e68641c28fbbad17fe83e667, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/c2e8488db66043bb8fb27d5d0861797a_SeqId_44_] to archive 2023-03-09 01:54:07,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:07,019 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/85c26ddfad0f46cdb38921c5b1644686 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/85c26ddfad0f46cdb38921c5b1644686 2023-03-09 01:54:07,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/191f0991666d4e808fd06e11154fb499 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/191f0991666d4e808fd06e11154fb499 2023-03-09 01:54:07,022 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/187b09cd1cc146639e31d6aa0a8dd2f6 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/187b09cd1cc146639e31d6aa0a8dd2f6 2023-03-09 01:54:07,023 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7585827f7c8c4693a5ab5acaab1d0a13 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7585827f7c8c4693a5ab5acaab1d0a13 2023-03-09 01:54:07,025 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/b9045dce73224be6b6128cd2195aa5e8 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/b9045dce73224be6b6128cd2195aa5e8 2023-03-09 01:54:07,026 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/2026a0d5c99b4a8cbc855b66d962ebd2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/2026a0d5c99b4a8cbc855b66d962ebd2 2023-03-09 01:54:07,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/0468dd7668f347a5b77159327f5fbb77 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/0468dd7668f347a5b77159327f5fbb77 2023-03-09 01:54:07,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/3b83ff0130a045299864452ef3441f62 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/3b83ff0130a045299864452ef3441f62 2023-03-09 01:54:07,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7cc79fb9e68641c28fbbad17fe83e667 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7cc79fb9e68641c28fbbad17fe83e667 2023-03-09 01:54:07,032 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/c2e8488db66043bb8fb27d5d0861797a_SeqId_44_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/c2e8488db66043bb8fb27d5d0861797a_SeqId_44_ 2023-03-09 01:54:07,453 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130fd10c283f58f4b87948bdb6d76d294b0 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130fd10c283f58f4b87948bdb6d76d294b0 2023-03-09 01:54:07,454 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:54:07,480 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b1c1509578c94117b62a456b66d067f9 first=Optional[r00] last=Optional[r8] 2023-03-09 01:54:07,481 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b1c1509578c94117b62a456b66d067f9}] 2023-03-09 01:54:07,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b1c1509578c94117b62a456b66d067f9 for inclusion in 519ef0416144e355c478496bd9b26e48/family1 2023-03-09 01:54:07,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-09 01:54:07,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:07,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:07,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b1c1509578c94117b62a456b66d067f9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t8ec1b727ru99gvs1e45gp1usaomh4j0pl27op37gdp8qq6ufu0a6ak9vrqufoj8/family1/b1c1509578c94117b62a456b66d067f9 2023-03-09 01:54:07,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t8ec1b727ru99gvs1e45gp1usaomh4j0pl27op37gdp8qq6ufu0a6ak9vrqufoj8/family1/b1c1509578c94117b62a456b66d067f9 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7d021b602dfc4156b4d284f7c7234787_SeqId_49_ 2023-03-09 01:54:07,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t8ec1b727ru99gvs1e45gp1usaomh4j0pl27op37gdp8qq6ufu0a6ak9vrqufoj8/family1/b1c1509578c94117b62a456b66d067f9 into 519ef0416144e355c478496bd9b26e48/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7d021b602dfc4156b4d284f7c7234787_SeqId_49_ - updating store file list. 2023-03-09 01:54:07,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7d021b602dfc4156b4d284f7c7234787_SeqId_49_ into 519ef0416144e355c478496bd9b26e48/family1 2023-03-09 01:54:07,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t8ec1b727ru99gvs1e45gp1usaomh4j0pl27op37gdp8qq6ufu0a6ak9vrqufoj8/family1/b1c1509578c94117b62a456b66d067f9 into 519ef0416144e355c478496bd9b26e48/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7d021b602dfc4156b4d284f7c7234787_SeqId_49_) 2023-03-09 01:54:07,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t8ec1b727ru99gvs1e45gp1usaomh4j0pl27op37gdp8qq6ufu0a6ak9vrqufoj8/family1/b1c1509578c94117b62a456b66d067f9 2023-03-09 01:54:07,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-09 01:54:07,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-09 01:54:07,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 519ef0416144e355c478496bd9b26e48/family1 is initiating minor compaction (all files) 2023-03-09 01:54:07,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@14759696; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:07,498 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 519ef0416144e355c478496bd9b26e48/family1 in testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:54:07,498 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/58e82328cb1c4b7884a095a235046224, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/27d2c85f4151499a856c9daff44adf0f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7d021b602dfc4156b4d284f7c7234787_SeqId_49_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp, totalSize=344.3 K 2023-03-09 01:54:07,498 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 58e82328cb1c4b7884a095a235046224, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843359 2023-03-09 01:54:07,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 27d2c85f4151499a856c9daff44adf0f, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243359 2023-03-09 01:54:07,499 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965243359 2023-03-09 01:54:07,500 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:54:07,500 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 7d021b602dfc4156b4d284f7c7234787_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014843359 2023-03-09 01:54:07,500 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:07,502 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120c031900904d44eeca15ba53221b27d1a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120c031900904d44eeca15ba53221b27d1a 2023-03-09 01:54:07,503 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511286ffd5655757e49abbaee90a909b856c7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511286ffd5655757e49abbaee90a909b856c7 2023-03-09 01:54:07,504 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113025fab766ab19498f83d6cc8926821ce4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113025fab766ab19498f83d6cc8926821ce4 2023-03-09 01:54:07,504 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-09 01:54:07,504 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-09 01:54:07,505 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:54:07,505 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:54:07,505 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:54:07,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:54:07,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:07,506 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:07,508 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9150 sec 2023-03-09 01:54:07,522 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 519ef0416144e355c478496bd9b26e48#family1#compaction#23 average throughput is 15.56 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:54:07,610 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=264 (was 247) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45348 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x53e7b61f sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/482268361.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45344 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45414 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7a6383b9-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-984539750_15 at /127.0.0.1:45508 [Receiving block BP-1290059279-172.31.14.131-1678326815090:blk_1073741900_1076] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45296 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7a6383b9-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7a6383b9-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:40738 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45384 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45494 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/2189e206cc5c4e21b2e03bc6b5eede60 block BP-1290059279-172.31.14.131-1678326815090:blk_1073741900_1076 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45258 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45332 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45306 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45386 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x27f12041-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7a6383b9-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7a6383b9-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/cluster_c7ae5eb2-da64-f932-cf16-ebb509eb1321/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7a6383b9-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45294 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7a6383b9-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113093960286f9384e98919561269dcd3bde_del java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45428 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45398 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-984539750_15 at /127.0.0.1:40756 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x27f12041-shared-pool-22 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x53e7b61f-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ResponseProcessor for block BP-1290059279-172.31.14.131-1678326815090:blk_1073741900_1076 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50548@0x53e7b61f-SendThread(127.0.0.1:50548) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: Thread-487 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: PacketResponder: BP-1290059279-172.31.14.131-1678326815090:blk_1073741900_1076, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45400 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1210834570_15 at /127.0.0.1:45280 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=620 (was 591) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=252 (was 289), ProcessCount=172 (was 173), AvailableMemoryMB=8599 (was 8070) - AvailableMemoryMB LEAK? - 2023-03-09 01:54:07,622 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=264, OpenFileDescriptor=620, MaxFileDescriptor=60000, SystemLoadAverage=252, ProcessCount=172, AvailableMemoryMB=8599 2023-03-09 01:54:07,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-09 01:54:07,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:07,627 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:54:07,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-09 01:54:07,628 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:54:07,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-09 01:54:07,630 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:07,630 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764 empty. 2023-03-09 01:54:07,631 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:07,631 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-09 01:54:07,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-09 01:54:07,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-09 01:54:08,043 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-09 01:54:08,045 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2c8c3d2b24a65eed3f072e8124247764, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:54:08,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-09 01:54:08,451 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:08,451 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 2c8c3d2b24a65eed3f072e8124247764, disabling compactions & flushes 2023-03-09 01:54:08,451 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:08,451 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:08,452 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. after waiting 0 ms 2023-03-09 01:54:08,452 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:08,452 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:08,452 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:08,454 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:54:08,455 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678326848455"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326848455"}]},"ts":"1678326848455"} 2023-03-09 01:54:08,457 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:54:08,458 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:54:08,458 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326848458"}]},"ts":"1678326848458"} 2023-03-09 01:54:08,459 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-09 01:54:08,467 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=2c8c3d2b24a65eed3f072e8124247764, ASSIGN}] 2023-03-09 01:54:08,468 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=2c8c3d2b24a65eed3f072e8124247764, ASSIGN 2023-03-09 01:54:08,469 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=2c8c3d2b24a65eed3f072e8124247764, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:54:08,620 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=2c8c3d2b24a65eed3f072e8124247764, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:08,621 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678326848620"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326848620"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326848620"}]},"ts":"1678326848620"} 2023-03-09 01:54:08,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 2c8c3d2b24a65eed3f072e8124247764, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:08,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-09 01:54:08,744 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/.tmp/family1/2189e206cc5c4e21b2e03bc6b5eede60 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/2189e206cc5c4e21b2e03bc6b5eede60 2023-03-09 01:54:08,750 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 519ef0416144e355c478496bd9b26e48/family1 of 519ef0416144e355c478496bd9b26e48 into 2189e206cc5c4e21b2e03bc6b5eede60(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:54:08,750 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:54:08,750 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1678326847497; duration=1sec 2023-03-09 01:54:08,750 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:08,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:08,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2c8c3d2b24a65eed3f072e8124247764, NAME => 'testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:54:08,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:08,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:08,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:08,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:08,780 INFO [StoreOpener-2c8c3d2b24a65eed3f072e8124247764-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:08,781 DEBUG [StoreOpener-2c8c3d2b24a65eed3f072e8124247764-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1 2023-03-09 01:54:08,781 DEBUG [StoreOpener-2c8c3d2b24a65eed3f072e8124247764-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1 2023-03-09 01:54:08,782 INFO [StoreOpener-2c8c3d2b24a65eed3f072e8124247764-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c8c3d2b24a65eed3f072e8124247764 columnFamilyName family1 2023-03-09 01:54:08,783 INFO [StoreOpener-2c8c3d2b24a65eed3f072e8124247764-1] regionserver.HStore(310): Store=2c8c3d2b24a65eed3f072e8124247764/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:08,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:08,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:08,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:54:08,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:54:08,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2c8c3d2b24a65eed3f072e8124247764; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69777524, jitterRate=0.039766132831573486}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:54:08,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:08,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764., pid=25, masterSystemTime=1678326848775 2023-03-09 01:54:08,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:08,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:08,792 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=2c8c3d2b24a65eed3f072e8124247764, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:08,793 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678326848792"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326848792"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326848792"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326848792"}]},"ts":"1678326848792"} 2023-03-09 01:54:08,796 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-09 01:54:08,796 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 2c8c3d2b24a65eed3f072e8124247764, server=jenkins-hbase4.apache.org,46511,1678326819757 in 171 msec 2023-03-09 01:54:08,798 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-09 01:54:08,798 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=2c8c3d2b24a65eed3f072e8124247764, ASSIGN in 329 msec 2023-03-09 01:54:08,799 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:54:08,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326848799"}]},"ts":"1678326848799"} 2023-03-09 01:54:08,800 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-09 01:54:08,803 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:54:08,804 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1790 sec 2023-03-09 01:54:08,807 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/58e82328cb1c4b7884a095a235046224, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/27d2c85f4151499a856c9daff44adf0f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7d021b602dfc4156b4d284f7c7234787_SeqId_49_] to archive 2023-03-09 01:54:08,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:08,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/58e82328cb1c4b7884a095a235046224 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/58e82328cb1c4b7884a095a235046224 2023-03-09 01:54:08,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/27d2c85f4151499a856c9daff44adf0f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/27d2c85f4151499a856c9daff44adf0f 2023-03-09 01:54:08,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/93e096f2c4a34ed1b5de7b6ed823ad78_SeqId_46_ 2023-03-09 01:54:08,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7d021b602dfc4156b4d284f7c7234787_SeqId_49_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/family1/7d021b602dfc4156b4d284f7c7234787_SeqId_49_ 2023-03-09 01:54:09,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-09 01:54:09,735 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-09 01:54:09,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:09,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:09,762 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:09,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:09,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:09,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:09,763 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:09,763 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:09,764 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:09,764 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:09,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:09,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:09,765 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:09,765 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:09,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:09,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:09,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:09,766 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:09,766 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:09,767 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-09 01:54:10,189 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a60b589de15340218aa6e6b0a4d41cbd to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130a60b589de15340218aa6e6b0a4d41cbd 2023-03-09 01:54:10,593 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/a7dc988e1eee43d38aaa2d1f9859d867 2023-03-09 01:54:10,599 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/a7dc988e1eee43d38aaa2d1f9859d867 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/a7dc988e1eee43d38aaa2d1f9859d867 2023-03-09 01:54:10,603 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/a7dc988e1eee43d38aaa2d1f9859d867, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-09 01:54:10,604 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 837ms, sequenceid=5, compaction requested=false 2023-03-09 01:54:10,605 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:10,605 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:10,605 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:10,605 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:10,608 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,608 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:10,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:10,608 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,608 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:10,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:10,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:10,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:10,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:10,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22da4b8b[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:10,609 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:10,610 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,611 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,611 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:10,611 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:10,611 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:10,611 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,611 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:10,612 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:10,612 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,612 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:10,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:10,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:10,612 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:10,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:10,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:10,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:10,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:10,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:10,614 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:10,614 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:10,614 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:10,614 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,614 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:10,616 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,616 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,616 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:10,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:10,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:10,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:10,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:10,616 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:10,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:10,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,618 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:10,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,621 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,621 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:10,621 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,621 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:10,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:10,621 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:10,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:10,621 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:10,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:10,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:10,621 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:10,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:10,622 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,622 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,622 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,622 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:10,622 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:10,722 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:10,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:10,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:10,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:10,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,729 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:10,729 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:10,730 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:10,730 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:10,730 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,730 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,731 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:10,731 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:10,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:10,732 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:10,732 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:10,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:10,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:10,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:10,733 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:10,733 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:10,733 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:10,733 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:10,733 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:10,733 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:10,733 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:10,733 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-09 01:54:11,146 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128264c198b001b4d2e9603d083a0c5e476 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128264c198b001b4d2e9603d083a0c5e476 2023-03-09 01:54:11,151 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/be8ea38e884c493fbefcbfe558cc1273 2023-03-09 01:54:11,157 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/be8ea38e884c493fbefcbfe558cc1273 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/be8ea38e884c493fbefcbfe558cc1273 2023-03-09 01:54:11,162 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/be8ea38e884c493fbefcbfe558cc1273, entries=1, sequenceid=9, filesize=5.0 K 2023-03-09 01:54:11,163 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 430ms, sequenceid=9, compaction requested=false 2023-03-09 01:54:11,164 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:11,164 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:11,164 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:11,164 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:11,164 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,164 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:11,164 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:11,166 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,166 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:11,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:11,166 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,166 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:11,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:11,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:11,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:11,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:11,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71a9b48c[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:11,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:11,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,169 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,170 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:11,170 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:11,170 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:11,170 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,170 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:11,171 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:11,171 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,171 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:11,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:11,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:11,171 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:11,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:11,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:11,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:11,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:11,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:11,174 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:11,174 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:11,174 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:11,174 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,174 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:11,176 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,176 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,177 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:11,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:11,177 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:11,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:11,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,178 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,181 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,181 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:11,181 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,181 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:11,181 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:11,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:11,181 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:11,182 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:11,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:11,182 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:11,182 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,182 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,182 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:11,182 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:11,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:11,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:11,282 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:11,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:11,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:11,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:11,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,290 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:11,290 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:11,290 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:11,290 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:11,290 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,290 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,292 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:11,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:11,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:11,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:11,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:11,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:11,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,293 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:11,293 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:11,293 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:11,293 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:11,293 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:11,293 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:11,293 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:11,294 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:11,706 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201fd6e00ac77cf412f8989819ed06a2709 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201fd6e00ac77cf412f8989819ed06a2709 2023-03-09 01:54:11,712 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/c3de2e2645a54a0883bbf92e1c5f6a4e 2023-03-09 01:54:11,717 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/c3de2e2645a54a0883bbf92e1c5f6a4e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c3de2e2645a54a0883bbf92e1c5f6a4e 2023-03-09 01:54:11,721 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c3de2e2645a54a0883bbf92e1c5f6a4e, entries=1, sequenceid=13, filesize=4.9 K 2023-03-09 01:54:11,722 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 428ms, sequenceid=13, compaction requested=true 2023-03-09 01:54:11,723 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:11,723 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:11,723 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:11,723 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:11,723 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,723 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:11,723 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:11,724 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,724 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:11,725 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,725 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:11,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:11,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:11,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16fd1798[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:11,726 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:11,726 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,728 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,728 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:11,728 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:11,728 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:11,728 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,728 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:11,730 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:11,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,730 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:11,730 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:11,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:11,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:11,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:11,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:11,732 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:11,732 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:11,732 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:11,732 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,732 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:11,734 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,734 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,734 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:11,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:11,734 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:11,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,735 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:11,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,741 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:11,741 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:11,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:11,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:11,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:11,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:11,742 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:11,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:11,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:11,742 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:11,842 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:11,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:11,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:11,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:11,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,849 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:11,849 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:11,849 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:11,849 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:11,850 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,850 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,851 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,851 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:11,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:11,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:11,852 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:11,852 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:11,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,852 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:11,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,853 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:11,853 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:11,853 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:11,853 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:11,853 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:11,853 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:11,853 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:11,853 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:11,853 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:12,267 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120597dc22f910ae42faa5a1bb0c7aa586e4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120597dc22f910ae42faa5a1bb0c7aa586e4 2023-03-09 01:54:12,672 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/c7fa8c6bb1ea4421aa57aa5a46a31516 2023-03-09 01:54:12,678 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/c7fa8c6bb1ea4421aa57aa5a46a31516 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c7fa8c6bb1ea4421aa57aa5a46a31516 2023-03-09 01:54:12,683 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c7fa8c6bb1ea4421aa57aa5a46a31516, entries=1, sequenceid=17, filesize=4.9 K 2023-03-09 01:54:12,684 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 831ms, sequenceid=17, compaction requested=true 2023-03-09 01:54:12,684 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:12,684 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:12,686 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,686 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:12,686 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,686 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@646b7a8f[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:12,688 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:12,688 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,690 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,690 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:12,691 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:12,691 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,691 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:12,691 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:12,692 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:12,692 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,692 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,692 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:12,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:12,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:12,694 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:12,694 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:12,694 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:12,694 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,694 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:12,696 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,696 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,696 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:12,696 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:12,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,697 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:12,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,700 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,700 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:12,700 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:12,700 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:12,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:12,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:12,700 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:12,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:12,700 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,701 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,701 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,701 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,701 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,701 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:12,701 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:12,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:12,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:12,801 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:12,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:12,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:12,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:12,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,808 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:12,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:12,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:12,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:12,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,811 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:12,811 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:12,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:12,811 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:12,811 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:12,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:12,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:12,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:12,813 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:12,813 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:12,813 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:12,813 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:12,813 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:12,813 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:12,813 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:12,813 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:13,231 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228e5a28a6fc73b4c888586ad40312c73b9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228e5a28a6fc73b4c888586ad40312c73b9 2023-03-09 01:54:13,637 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/69542bb16e5449318b0c979f103efa7a 2023-03-09 01:54:13,642 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/69542bb16e5449318b0c979f103efa7a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/69542bb16e5449318b0c979f103efa7a 2023-03-09 01:54:13,646 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/69542bb16e5449318b0c979f103efa7a, entries=1, sequenceid=21, filesize=4.9 K 2023-03-09 01:54:13,647 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 834ms, sequenceid=21, compaction requested=true 2023-03-09 01:54:13,647 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:13,647 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:13,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,649 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:13,650 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,650 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@149de39a[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:13,651 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:13,651 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,652 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,652 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:13,652 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:13,652 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:13,652 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,652 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:13,654 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:13,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:13,656 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:13,656 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:13,656 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:13,656 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,656 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:13,658 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,658 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:13,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:13,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,658 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:13,659 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:13,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:13,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:13,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:13,660 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:13,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,662 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,662 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,662 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:13,662 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:13,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:13,662 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:13,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:13,662 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:13,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:13,662 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:13,663 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:13,663 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:13,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:13,662 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:13,663 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,663 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,763 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:13,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:13,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:13,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:13,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,770 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:13,770 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:13,770 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:13,770 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:13,771 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,771 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,772 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:13,772 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:13,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:13,772 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:13,772 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:13,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,773 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:13,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,773 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,773 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:13,773 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:13,773 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:13,773 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:13,773 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:13,774 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:13,774 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:13,774 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:13,774 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:13,774 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:14,192 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231351e6ecbc6774f9c80ac0119cdd9af26 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231351e6ecbc6774f9c80ac0119cdd9af26 2023-03-09 01:54:14,197 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/cbb92133f58d4f9faac6791d0cae0ace 2023-03-09 01:54:14,202 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/cbb92133f58d4f9faac6791d0cae0ace as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/cbb92133f58d4f9faac6791d0cae0ace 2023-03-09 01:54:14,207 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/cbb92133f58d4f9faac6791d0cae0ace, entries=1, sequenceid=25, filesize=4.9 K 2023-03-09 01:54:14,208 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 434ms, sequenceid=25, compaction requested=true 2023-03-09 01:54:14,209 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:14,209 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:14,209 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:14,210 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,211 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:14,211 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,211 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:14,212 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28ac3851[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:14,213 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,220 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,220 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,220 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:14,220 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:14,220 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:14,220 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:14,222 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,222 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:14,222 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:14,222 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:14,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:14,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:14,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:14,224 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:14,224 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:14,224 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:14,224 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:14,224 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,224 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:14,229 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,229 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,229 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:14,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:14,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:14,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:14,230 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:14,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,231 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:14,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,233 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,233 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:14,233 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,233 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:14,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:14,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:14,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:14,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:14,233 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:14,234 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:14,234 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:14,233 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:14,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:14,234 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,234 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,234 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,234 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,313 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:54:14,334 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:14,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:14,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:14,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:14,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,341 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:14,341 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:14,341 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:14,341 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:14,342 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,342 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,343 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:14,343 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:14,343 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,344 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:14,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:14,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:14,345 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:14,345 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:14,345 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:14,759 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101f427fdbe6fd045f899770be2ac851833 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101f427fdbe6fd045f899770be2ac851833 2023-03-09 01:54:15,165 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/c2565951ea8f4c32a05c81fff609ebd9 2023-03-09 01:54:15,171 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/c2565951ea8f4c32a05c81fff609ebd9 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c2565951ea8f4c32a05c81fff609ebd9 2023-03-09 01:54:15,175 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c2565951ea8f4c32a05c81fff609ebd9, entries=1, sequenceid=29, filesize=4.9 K 2023-03-09 01:54:15,176 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 831ms, sequenceid=29, compaction requested=true 2023-03-09 01:54:15,176 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:15,176 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:15,176 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:15,176 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:15,176 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,176 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:15,176 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:15,183 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,183 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:15,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:15,183 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:15,183 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:15,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:15,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5693ab0b[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:15,184 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:15,185 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,191 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,191 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:15,191 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:15,191 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,191 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:15,191 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:15,199 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:15,199 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,199 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:15,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,199 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:15,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:15,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:15,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:15,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:15,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:15,201 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:15,201 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:15,201 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:15,201 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,201 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:15,208 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,208 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,208 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:15,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:15,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:15,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:15,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:15,208 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:15,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:15,209 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:15,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,213 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,213 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:15,213 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,213 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:15,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:15,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:15,213 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:15,213 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:15,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:15,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:15,213 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:15,213 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,214 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:15,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:15,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:15,214 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,214 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,214 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,314 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:15,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:15,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:15,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:15,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,321 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:15,321 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:15,322 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:15,322 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:15,322 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,322 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,323 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:15,323 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:15,324 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:15,324 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,324 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,325 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,325 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:15,325 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:15,325 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:15,325 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:15,325 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:15,325 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:15,325 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:15,325 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:15,325 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:15,325 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:15,738 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032de25641ef4147b39cf4ef85f68bfae9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601032de25641ef4147b39cf4ef85f68bfae9 2023-03-09 01:54:16,144 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/9723b0766ba1493bbf9f7a6b9b8c883e 2023-03-09 01:54:16,150 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/9723b0766ba1493bbf9f7a6b9b8c883e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/9723b0766ba1493bbf9f7a6b9b8c883e 2023-03-09 01:54:16,155 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/9723b0766ba1493bbf9f7a6b9b8c883e, entries=1, sequenceid=33, filesize=4.9 K 2023-03-09 01:54:16,155 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 830ms, sequenceid=33, compaction requested=true 2023-03-09 01:54:16,156 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:16,156 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:16,156 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:16,156 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:16,156 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,156 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:16,156 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:16,158 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,158 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:16,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:16,158 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,158 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:16,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:16,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69131c54[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:16,159 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:16,159 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,161 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,161 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:16,161 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:16,161 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:16,161 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,161 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:16,164 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:16,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:16,166 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:16,166 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:16,166 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:16,166 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,166 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:16,169 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,169 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:16,169 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,170 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:16,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:16,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,171 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:16,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:16,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,173 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:16,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:16,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:16,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:16,173 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:16,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:16,174 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,274 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:16,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:16,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:16,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:16,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,281 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:16,281 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:16,281 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:16,281 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:16,281 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,281 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,283 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:16,283 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:16,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:16,283 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:16,283 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:16,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:16,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,284 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,284 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:16,284 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:16,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:16,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:16,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:16,285 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:16,285 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:16,285 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:16,285 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:16,285 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:16,699 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509071dbe4f69aa934175ba5e8b5223c9fe7e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509071dbe4f69aa934175ba5e8b5223c9fe7e 2023-03-09 01:54:17,105 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/e10ec167bd044bd297e2671b30bacdc9 2023-03-09 01:54:17,110 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/e10ec167bd044bd297e2671b30bacdc9 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/e10ec167bd044bd297e2671b30bacdc9 2023-03-09 01:54:17,115 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/e10ec167bd044bd297e2671b30bacdc9, entries=1, sequenceid=37, filesize=4.9 K 2023-03-09 01:54:17,116 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 831ms, sequenceid=37, compaction requested=true 2023-03-09 01:54:17,117 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:17,117 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:17,117 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:17,120 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,120 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:17,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:17,120 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,120 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:17,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:17,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:17,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:17,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:17,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1269a3df[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:17,122 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:17,122 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,125 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,126 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:17,126 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:17,126 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:17,126 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,126 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:17,128 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,128 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:17,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:17,128 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:17,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:17,128 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:17,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:17,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:17,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:17,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:17,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:17,130 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:17,130 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:17,130 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:17,130 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,130 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:17,132 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,132 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,132 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:17,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,133 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:17,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:17,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:17,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:17,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,134 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:17,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,136 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,136 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:17,136 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:17,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:17,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:17,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:17,136 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,136 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:17,136 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:17,137 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:17,137 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:17,137 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:17,136 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:17,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:17,137 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,137 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,137 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,237 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:17,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:17,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:17,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:17,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,244 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:17,244 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:17,244 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:17,244 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:17,245 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,245 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,248 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,248 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:17,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:17,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:17,248 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:17,248 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:17,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:17,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:17,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:17,249 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:17,249 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:17,249 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:17,249 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:17,249 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:17,249 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:17,249 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. started... 2023-03-09 01:54:17,250 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c8c3d2b24a65eed3f072e8124247764 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:17,663 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511206fec5ba9b17c4493bd1ee2953cdb0d6e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511206fec5ba9b17c4493bd1ee2953cdb0d6e 2023-03-09 01:54:18,069 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/4adaf76b98c3463c9855ab1b031eee3c 2023-03-09 01:54:18,075 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/4adaf76b98c3463c9855ab1b031eee3c as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/4adaf76b98c3463c9855ab1b031eee3c 2023-03-09 01:54:18,080 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/4adaf76b98c3463c9855ab1b031eee3c, entries=1, sequenceid=41, filesize=4.9 K 2023-03-09 01:54:18,081 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c8c3d2b24a65eed3f072e8124247764 in 831ms, sequenceid=41, compaction requested=true 2023-03-09 01:54:18,081 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:18,081 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:18,082 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:18,082 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:18,082 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,082 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:18,082 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,083 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:18,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:18,084 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,084 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:18,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:18,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:18,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:18,085 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:18,085 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b4e6f73[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:18,085 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:18,086 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,087 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,087 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:18,087 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:18,087 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,087 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:18,087 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:18,089 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:18,089 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:18,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:18,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:18,092 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:18,092 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:18,092 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:18,092 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,092 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:18,094 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,094 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:18,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,094 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:18,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:18,095 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,096 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:18,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,099 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,099 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:18,099 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,099 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:18,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:18,099 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-09 01:54:18,099 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:18,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:18,099 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:18,099 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:18,100 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,100 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,100 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:18,100 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:18,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:18,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:18,200 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:18,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:18,203 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-09 01:54:18,228 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:54:18,229 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x53e7b61f to 127.0.0.1:50548 2023-03-09 01:54:18,229 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:54:18,229 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:18,230 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:18,231 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:54:18,231 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6aa3b364 to 127.0.0.1:50548 2023-03-09 01:54:18,231 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:54:18,232 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:54:18,232 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x41e2de52 to 127.0.0.1:50548 2023-03-09 01:54:18,232 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:54:18,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:18,237 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327458237 2023-03-09 01:54:18,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-09 01:54:18,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-09 01:54:18,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:18,240 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x32a91364 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:54:18,250 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ddc7edd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:54:18,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,097 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103098c17c2c4414ac0ae674915ea4acb27 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103098c17c2c4414ac0ae674915ea4acb27 2023-03-09 01:54:19,098 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:19,115 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:54:19,116 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44002, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:54:19,127 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:54:19,130 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60694, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:54:19,158 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f1675f85688d4cd3b1f146ac5c8ac2a0 first=Optional[r3] last=Optional[r6] 2023-03-09 01:54:19,159 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f1675f85688d4cd3b1f146ac5c8ac2a0}] 2023-03-09 01:54:19,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f1675f85688d4cd3b1f146ac5c8ac2a0 for inclusion in 2c8c3d2b24a65eed3f072e8124247764/family1 2023-03-09 01:54:19,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-09 01:54:19,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:19,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:19,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f1675f85688d4cd3b1f146ac5c8ac2a0 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7i8i6sadlg34ddrbst6v89b3p2ghbnnbe4jav60fjn4om9jec6vq7el93n5q0l4/family1/f1675f85688d4cd3b1f146ac5c8ac2a0 2023-03-09 01:54:19,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7i8i6sadlg34ddrbst6v89b3p2ghbnnbe4jav60fjn4om9jec6vq7el93n5q0l4/family1/f1675f85688d4cd3b1f146ac5c8ac2a0 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/39a17e8187f8480896976a3ad8ac28bf_SeqId_44_ 2023-03-09 01:54:19,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7i8i6sadlg34ddrbst6v89b3p2ghbnnbe4jav60fjn4om9jec6vq7el93n5q0l4/family1/f1675f85688d4cd3b1f146ac5c8ac2a0 into 2c8c3d2b24a65eed3f072e8124247764/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/39a17e8187f8480896976a3ad8ac28bf_SeqId_44_ - updating store file list. 2023-03-09 01:54:19,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/39a17e8187f8480896976a3ad8ac28bf_SeqId_44_ into 2c8c3d2b24a65eed3f072e8124247764/family1 2023-03-09 01:54:19,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7i8i6sadlg34ddrbst6v89b3p2ghbnnbe4jav60fjn4om9jec6vq7el93n5q0l4/family1/f1675f85688d4cd3b1f146ac5c8ac2a0 into 2c8c3d2b24a65eed3f072e8124247764/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/39a17e8187f8480896976a3ad8ac28bf_SeqId_44_) 2023-03-09 01:54:19,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7i8i6sadlg34ddrbst6v89b3p2ghbnnbe4jav60fjn4om9jec6vq7el93n5q0l4/family1/f1675f85688d4cd3b1f146ac5c8ac2a0 2023-03-09 01:54:19,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-09 01:54:19,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-09 01:54:19,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 2c8c3d2b24a65eed3f072e8124247764/family1 is initiating minor compaction 2023-03-09 01:54:19,196 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2c8c3d2b24a65eed3f072e8124247764/family1 in testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:54:19,196 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/be8ea38e884c493fbefcbfe558cc1273, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c3de2e2645a54a0883bbf92e1c5f6a4e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c7fa8c6bb1ea4421aa57aa5a46a31516, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/69542bb16e5449318b0c979f103efa7a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/cbb92133f58d4f9faac6791d0cae0ace, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c2565951ea8f4c32a05c81fff609ebd9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/9723b0766ba1493bbf9f7a6b9b8c883e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/e10ec167bd044bd297e2671b30bacdc9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/4adaf76b98c3463c9855ab1b031eee3c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/39a17e8187f8480896976a3ad8ac28bf_SeqId_44_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp, totalSize=50.0 K 2023-03-09 01:54:19,196 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting be8ea38e884c493fbefcbfe558cc1273, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting c3de2e2645a54a0883bbf92e1c5f6a4e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting c7fa8c6bb1ea4421aa57aa5a46a31516, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 69542bb16e5449318b0c979f103efa7a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting cbb92133f58d4f9faac6791d0cae0ace, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting c2565951ea8f4c32a05c81fff609ebd9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 9723b0766ba1493bbf9f7a6b9b8c883e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting e10ec167bd044bd297e2671b30bacdc9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 4adaf76b98c3463c9855ab1b031eee3c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-09 01:54:19,197 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 39a17e8187f8480896976a3ad8ac28bf_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-09 01:54:19,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6034240f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:19,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:19,204 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:19,206 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228e5a28a6fc73b4c888586ad40312c73b9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228e5a28a6fc73b4c888586ad40312c73b9 2023-03-09 01:54:19,212 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231351e6ecbc6774f9c80ac0119cdd9af26 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231351e6ecbc6774f9c80ac0119cdd9af26 2023-03-09 01:54:19,214 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101f427fdbe6fd045f899770be2ac851833 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101f427fdbe6fd045f899770be2ac851833 2023-03-09 01:54:19,215 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601032de25641ef4147b39cf4ef85f68bfae9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601032de25641ef4147b39cf4ef85f68bfae9 2023-03-09 01:54:19,216 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-09 01:54:19,216 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-09 01:54:19,216 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,216 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,216 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,217 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,217 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,217 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,217 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,217 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,265 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2c8c3d2b24a65eed3f072e8124247764#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:54:19,282 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c14ac1f331044460b39b530ab9267a60 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205c14ac1f331044460b39b530ab9267a60 2023-03-09 01:54:19,283 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:19,309 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/862db6862c174c33b8593227886e7a85 first=Optional[r1] last=Optional[r2] 2023-03-09 01:54:19,310 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/862db6862c174c33b8593227886e7a85}] 2023-03-09 01:54:19,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/862db6862c174c33b8593227886e7a85 for inclusion in 2c8c3d2b24a65eed3f072e8124247764/family1 2023-03-09 01:54:19,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-09 01:54:19,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:19,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:19,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/862db6862c174c33b8593227886e7a85 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__5i29oclghj1o3v1u28r6mgrqroa624qfc6f99c5j143mpgm0bs2n5iaee1vpqc6f/family1/862db6862c174c33b8593227886e7a85 2023-03-09 01:54:19,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__5i29oclghj1o3v1u28r6mgrqroa624qfc6f99c5j143mpgm0bs2n5iaee1vpqc6f/family1/862db6862c174c33b8593227886e7a85 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/8bea27e3092c4b65aec7a7aa5202f1cd_SeqId_46_ 2023-03-09 01:54:19,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__5i29oclghj1o3v1u28r6mgrqroa624qfc6f99c5j143mpgm0bs2n5iaee1vpqc6f/family1/862db6862c174c33b8593227886e7a85 into 2c8c3d2b24a65eed3f072e8124247764/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/8bea27e3092c4b65aec7a7aa5202f1cd_SeqId_46_ - updating store file list. 2023-03-09 01:54:19,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/8bea27e3092c4b65aec7a7aa5202f1cd_SeqId_46_ into 2c8c3d2b24a65eed3f072e8124247764/family1 2023-03-09 01:54:19,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__5i29oclghj1o3v1u28r6mgrqroa624qfc6f99c5j143mpgm0bs2n5iaee1vpqc6f/family1/862db6862c174c33b8593227886e7a85 into 2c8c3d2b24a65eed3f072e8124247764/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/8bea27e3092c4b65aec7a7aa5202f1cd_SeqId_46_) 2023-03-09 01:54:19,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithWeeklyPolicy__5i29oclghj1o3v1u28r6mgrqroa624qfc6f99c5j143mpgm0bs2n5iaee1vpqc6f/family1/862db6862c174c33b8593227886e7a85 2023-03-09 01:54:19,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:19,333 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:19,335 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201fd6e00ac77cf412f8989819ed06a2709 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201fd6e00ac77cf412f8989819ed06a2709 2023-03-09 01:54:19,336 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120597dc22f910ae42faa5a1bb0c7aa586e4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120597dc22f910ae42faa5a1bb0c7aa586e4 2023-03-09 01:54:19,336 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-09 01:54:19,336 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-09 01:54:19,337 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:19,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:19,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:19,338 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:19,341 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.1110 sec 2023-03-09 01:54:19,480 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=255 (was 264), OpenFileDescriptor=608 (was 620), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=214 (was 252), ProcessCount=171 (was 172), AvailableMemoryMB=8599 (was 8599) 2023-03-09 01:54:19,494 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=255, OpenFileDescriptor=608, MaxFileDescriptor=60000, SystemLoadAverage=214, ProcessCount=171, AvailableMemoryMB=8589 2023-03-09 01:54:19,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-09 01:54:19,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:19,500 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:54:19,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-09 01:54:19,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-09 01:54:19,501 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:54:19,504 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:19,504 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:19,504 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:19,504 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0 empty. 2023-03-09 01:54:19,504 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c empty. 2023-03-09 01:54:19,504 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7 empty. 2023-03-09 01:54:19,505 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:19,505 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:19,505 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:19,505 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-09 01:54:19,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-09 01:54:19,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-09 01:54:19,917 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-09 01:54:19,919 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => eb9b94d3d4712803b80d37dce323940c, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:54:19,919 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => cd5a60b17949509277f0e081b9d03fe0, NAME => 'testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:54:19,919 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 51e7af8c834c7e1949ec8753f22a28d7, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:54:19,953 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:19,953 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing eb9b94d3d4712803b80d37dce323940c, disabling compactions & flushes 2023-03-09 01:54:19,953 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:19,953 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:19,953 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. after waiting 0 ms 2023-03-09 01:54:19,953 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:19,953 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:19,953 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for eb9b94d3d4712803b80d37dce323940c: 2023-03-09 01:54:19,961 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:19,961 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing cd5a60b17949509277f0e081b9d03fe0, disabling compactions & flushes 2023-03-09 01:54:19,961 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:19,961 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:19,961 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. after waiting 0 ms 2023-03-09 01:54:19,961 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:19,961 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:19,961 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for cd5a60b17949509277f0e081b9d03fe0: 2023-03-09 01:54:20,087 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/.tmp/family1/f123e468d03b4554952c6d2c5380846b as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/f123e468d03b4554952c6d2c5380846b 2023-03-09 01:54:20,093 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2c8c3d2b24a65eed3f072e8124247764/family1 of 2c8c3d2b24a65eed3f072e8124247764 into f123e468d03b4554952c6d2c5380846b(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:54:20,093 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:54:20,093 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678326859196; duration=0sec 2023-03-09 01:54:20,094 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:20,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-09 01:54:20,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/be8ea38e884c493fbefcbfe558cc1273, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c3de2e2645a54a0883bbf92e1c5f6a4e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c7fa8c6bb1ea4421aa57aa5a46a31516, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/69542bb16e5449318b0c979f103efa7a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/cbb92133f58d4f9faac6791d0cae0ace, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c2565951ea8f4c32a05c81fff609ebd9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/9723b0766ba1493bbf9f7a6b9b8c883e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/e10ec167bd044bd297e2671b30bacdc9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/4adaf76b98c3463c9855ab1b031eee3c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/39a17e8187f8480896976a3ad8ac28bf_SeqId_44_] to archive 2023-03-09 01:54:20,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:20,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/be8ea38e884c493fbefcbfe558cc1273 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/be8ea38e884c493fbefcbfe558cc1273 2023-03-09 01:54:20,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c3de2e2645a54a0883bbf92e1c5f6a4e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c3de2e2645a54a0883bbf92e1c5f6a4e 2023-03-09 01:54:20,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c7fa8c6bb1ea4421aa57aa5a46a31516 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c7fa8c6bb1ea4421aa57aa5a46a31516 2023-03-09 01:54:20,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/69542bb16e5449318b0c979f103efa7a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/69542bb16e5449318b0c979f103efa7a 2023-03-09 01:54:20,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/cbb92133f58d4f9faac6791d0cae0ace to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/cbb92133f58d4f9faac6791d0cae0ace 2023-03-09 01:54:20,131 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c2565951ea8f4c32a05c81fff609ebd9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/c2565951ea8f4c32a05c81fff609ebd9 2023-03-09 01:54:20,133 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/9723b0766ba1493bbf9f7a6b9b8c883e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/9723b0766ba1493bbf9f7a6b9b8c883e 2023-03-09 01:54:20,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/e10ec167bd044bd297e2671b30bacdc9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/e10ec167bd044bd297e2671b30bacdc9 2023-03-09 01:54:20,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/4adaf76b98c3463c9855ab1b031eee3c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/4adaf76b98c3463c9855ab1b031eee3c 2023-03-09 01:54:20,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/39a17e8187f8480896976a3ad8ac28bf_SeqId_44_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/family1/39a17e8187f8480896976a3ad8ac28bf_SeqId_44_ 2023-03-09 01:54:20,355 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:20,356 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 51e7af8c834c7e1949ec8753f22a28d7, disabling compactions & flushes 2023-03-09 01:54:20,356 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:20,356 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:20,356 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. after waiting 0 ms 2023-03-09 01:54:20,356 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:20,356 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:20,356 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 51e7af8c834c7e1949ec8753f22a28d7: 2023-03-09 01:54:20,359 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:54:20,359 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678326860359"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326860359"}]},"ts":"1678326860359"} 2023-03-09 01:54:20,360 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678326860359"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326860359"}]},"ts":"1678326860359"} 2023-03-09 01:54:20,360 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678326860359"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326860359"}]},"ts":"1678326860359"} 2023-03-09 01:54:20,363 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-09 01:54:20,364 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:54:20,364 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326860364"}]},"ts":"1678326860364"} 2023-03-09 01:54:20,365 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-09 01:54:20,368 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=cd5a60b17949509277f0e081b9d03fe0, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=eb9b94d3d4712803b80d37dce323940c, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=51e7af8c834c7e1949ec8753f22a28d7, ASSIGN}] 2023-03-09 01:54:20,370 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=eb9b94d3d4712803b80d37dce323940c, ASSIGN 2023-03-09 01:54:20,370 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=cd5a60b17949509277f0e081b9d03fe0, ASSIGN 2023-03-09 01:54:20,371 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=51e7af8c834c7e1949ec8753f22a28d7, ASSIGN 2023-03-09 01:54:20,371 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=eb9b94d3d4712803b80d37dce323940c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:54:20,371 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=cd5a60b17949509277f0e081b9d03fe0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:54:20,372 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=51e7af8c834c7e1949ec8753f22a28d7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:54:20,523 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=cd5a60b17949509277f0e081b9d03fe0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:20,523 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=51e7af8c834c7e1949ec8753f22a28d7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:20,523 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=eb9b94d3d4712803b80d37dce323940c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:20,523 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678326860523"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326860523"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326860523"}]},"ts":"1678326860523"} 2023-03-09 01:54:20,523 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678326860523"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326860523"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326860523"}]},"ts":"1678326860523"} 2023-03-09 01:54:20,523 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678326860523"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326860523"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326860523"}]},"ts":"1678326860523"} 2023-03-09 01:54:20,525 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure cd5a60b17949509277f0e081b9d03fe0, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:20,526 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure 51e7af8c834c7e1949ec8753f22a28d7, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:20,527 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure eb9b94d3d4712803b80d37dce323940c, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:20,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-09 01:54:20,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:20,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cd5a60b17949509277f0e081b9d03fe0, NAME => 'testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0.', STARTKEY => '', ENDKEY => '1'} 2023-03-09 01:54:20,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:20,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:20,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:20,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:20,683 INFO [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:20,685 DEBUG [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0/family1 2023-03-09 01:54:20,685 DEBUG [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0/family1 2023-03-09 01:54:20,686 INFO [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cd5a60b17949509277f0e081b9d03fe0 columnFamilyName family1 2023-03-09 01:54:20,687 INFO [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] regionserver.HStore(310): Store=cd5a60b17949509277f0e081b9d03fe0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:20,688 INFO [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:20,689 DEBUG [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0/family2 2023-03-09 01:54:20,689 DEBUG [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0/family2 2023-03-09 01:54:20,690 INFO [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cd5a60b17949509277f0e081b9d03fe0 columnFamilyName family2 2023-03-09 01:54:20,691 INFO [StoreOpener-cd5a60b17949509277f0e081b9d03fe0-1] regionserver.HStore(310): Store=cd5a60b17949509277f0e081b9d03fe0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:20,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:20,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:20,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:54:20,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:54:20,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:54:20,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cd5a60b17949509277f0e081b9d03fe0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67330551, jitterRate=0.0033033937215805054}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:54:20,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cd5a60b17949509277f0e081b9d03fe0: 2023-03-09 01:54:20,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0., pid=31, masterSystemTime=1678326860677 2023-03-09 01:54:20,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:20,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:20,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:20,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eb9b94d3d4712803b80d37dce323940c, NAME => 'testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c.', STARTKEY => '1', ENDKEY => '2'} 2023-03-09 01:54:20,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:20,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:20,702 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=cd5a60b17949509277f0e081b9d03fe0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:20,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:20,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:20,702 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678326860702"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326860702"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326860702"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326860702"}]},"ts":"1678326860702"} 2023-03-09 01:54:20,704 INFO [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:20,705 DEBUG [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c/family1 2023-03-09 01:54:20,705 DEBUG [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c/family1 2023-03-09 01:54:20,706 INFO [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb9b94d3d4712803b80d37dce323940c columnFamilyName family1 2023-03-09 01:54:20,706 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-03-09 01:54:20,706 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure cd5a60b17949509277f0e081b9d03fe0, server=jenkins-hbase4.apache.org,46511,1678326819757 in 178 msec 2023-03-09 01:54:20,707 INFO [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] regionserver.HStore(310): Store=eb9b94d3d4712803b80d37dce323940c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:20,707 INFO [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:20,708 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=cd5a60b17949509277f0e081b9d03fe0, ASSIGN in 338 msec 2023-03-09 01:54:20,709 DEBUG [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c/family2 2023-03-09 01:54:20,709 DEBUG [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c/family2 2023-03-09 01:54:20,710 INFO [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb9b94d3d4712803b80d37dce323940c columnFamilyName family2 2023-03-09 01:54:20,710 INFO [StoreOpener-eb9b94d3d4712803b80d37dce323940c-1] regionserver.HStore(310): Store=eb9b94d3d4712803b80d37dce323940c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:20,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:20,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:20,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:54:20,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:54:20,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:54:20,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eb9b94d3d4712803b80d37dce323940c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60723513, jitterRate=-0.09514914453029633}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:54:20,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eb9b94d3d4712803b80d37dce323940c: 2023-03-09 01:54:20,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c., pid=33, masterSystemTime=1678326860677 2023-03-09 01:54:20,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:20,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:20,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:20,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 51e7af8c834c7e1949ec8753f22a28d7, NAME => 'testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7.', STARTKEY => '2', ENDKEY => ''} 2023-03-09 01:54:20,720 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=eb9b94d3d4712803b80d37dce323940c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:20,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:20,720 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1678326860720"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326860720"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326860720"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326860720"}]},"ts":"1678326860720"} 2023-03-09 01:54:20,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:20,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:20,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:20,723 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-03-09 01:54:20,723 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure eb9b94d3d4712803b80d37dce323940c, server=jenkins-hbase4.apache.org,46511,1678326819757 in 194 msec 2023-03-09 01:54:20,723 INFO [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:20,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=eb9b94d3d4712803b80d37dce323940c, ASSIGN in 355 msec 2023-03-09 01:54:20,725 DEBUG [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1 2023-03-09 01:54:20,725 DEBUG [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1 2023-03-09 01:54:20,725 INFO [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51e7af8c834c7e1949ec8753f22a28d7 columnFamilyName family1 2023-03-09 01:54:20,726 INFO [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] regionserver.HStore(310): Store=51e7af8c834c7e1949ec8753f22a28d7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:20,727 INFO [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:20,728 DEBUG [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family2 2023-03-09 01:54:20,728 DEBUG [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family2 2023-03-09 01:54:20,728 INFO [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51e7af8c834c7e1949ec8753f22a28d7 columnFamilyName family2 2023-03-09 01:54:20,729 INFO [StoreOpener-51e7af8c834c7e1949ec8753f22a28d7-1] regionserver.HStore(310): Store=51e7af8c834c7e1949ec8753f22a28d7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:20,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:20,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:20,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:54:20,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:54:20,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:54:20,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 51e7af8c834c7e1949ec8753f22a28d7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73152691, jitterRate=0.0900600403547287}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:54:20,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 51e7af8c834c7e1949ec8753f22a28d7: 2023-03-09 01:54:20,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7., pid=32, masterSystemTime=1678326860677 2023-03-09 01:54:20,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:20,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:20,739 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=51e7af8c834c7e1949ec8753f22a28d7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:20,739 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1678326860739"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326860739"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326860739"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326860739"}]},"ts":"1678326860739"} 2023-03-09 01:54:20,742 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-03-09 01:54:20,742 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure 51e7af8c834c7e1949ec8753f22a28d7, server=jenkins-hbase4.apache.org,46511,1678326819757 in 214 msec 2023-03-09 01:54:20,744 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-03-09 01:54:20,744 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=51e7af8c834c7e1949ec8753f22a28d7, ASSIGN in 374 msec 2023-03-09 01:54:20,745 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:54:20,745 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326860745"}]},"ts":"1678326860745"} 2023-03-09 01:54:20,746 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-09 01:54:20,749 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:54:20,751 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2520 sec 2023-03-09 01:54:21,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-09 01:54:21,608 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-09 01:54:21,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:21,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:21,618 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-09 01:54:21,618 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:21,619 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-09 01:54:21,619 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-09 01:54:21,619 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:21,619 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:21,627 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:21,627 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:21,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:21,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:21,627 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:21,627 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:21,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:21,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:21,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:21,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:21,628 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:21,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:21,628 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-09 01:54:21,628 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:21,629 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-09 01:54:21,629 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:21,629 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. started... 2023-03-09 01:54:21,629 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:21,629 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:54:21,630 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cd5a60b17949509277f0e081b9d03fe0: 2023-03-09 01:54:21,630 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:21,630 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. started... 2023-03-09 01:54:21,629 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. started... 2023-03-09 01:54:21,630 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:21,630 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:21,631 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 51e7af8c834c7e1949ec8753f22a28d7 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-09 01:54:21,632 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:54:21,632 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for eb9b94d3d4712803b80d37dce323940c: 2023-03-09 01:54:21,632 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:21,632 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:54:21,653 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030987d299ef52cd47b3a4cb8a16af01f6a2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030987d299ef52cd47b3a4cb8a16af01f6a2 2023-03-09 01:54:22,062 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/.tmp/family1/870971919a7b469d8d48f115af9b9b51 2023-03-09 01:54:22,068 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/.tmp/family1/870971919a7b469d8d48f115af9b9b51 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/870971919a7b469d8d48f115af9b9b51 2023-03-09 01:54:22,083 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/870971919a7b469d8d48f115af9b9b51, entries=1, sequenceid=5, filesize=4.9 K 2023-03-09 01:54:22,084 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 51e7af8c834c7e1949ec8753f22a28d7 in 453ms, sequenceid=5, compaction requested=false 2023-03-09 01:54:22,084 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-09 01:54:22,085 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 51e7af8c834c7e1949ec8753f22a28d7: 2023-03-09 01:54:22,085 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:22,085 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:54:22,085 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:54:22,085 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,085 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-09 01:54:22,085 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-09 01:54:22,087 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,087 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:22,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:22,087 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,087 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:22,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:22,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:22,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:22,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-09 01:54:22,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@104e4ce2[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:22,089 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-09 01:54:22,089 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,090 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,091 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-09 01:54:22,091 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-09 01:54:22,091 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:22,091 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,091 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:22,092 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,092 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-09 01:54:22,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:22,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:22,092 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:22,093 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-09 01:54:22,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:22,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:22,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:22,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:22,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-09 01:54:22,095 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-09 01:54:22,095 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:22,095 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:22,095 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,095 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:22,097 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,097 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:22,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,098 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:22,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:22,098 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:22,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,100 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:22,105 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:22,105 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:22,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:22,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-09 01:54:22,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,105 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:22,106 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:22,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:22,106 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:22,206 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:22,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:22,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:22,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:22,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,213 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-09 01:54:22,214 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:22,214 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-09 01:54:22,214 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-09 01:54:22,214 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,214 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,215 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:22,215 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:22,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:22,216 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:22,216 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:22,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:22,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,217 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-09 01:54:22,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:22,217 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-09 01:54:22,217 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:22,217 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-09 01:54:22,217 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:22,218 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:54:22,218 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. started... 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cd5a60b17949509277f0e081b9d03fe0: 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. started... 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. started... 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:54:22,218 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 51e7af8c834c7e1949ec8753f22a28d7 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for eb9b94d3d4712803b80d37dce323940c: 2023-03-09 01:54:22,218 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:54:22,218 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:54:22,218 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:54:22,639 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309bed500ab33ff4303a8d509b9ee5ff054 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230309bed500ab33ff4303a8d509b9ee5ff054 2023-03-09 01:54:23,047 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/.tmp/family1/2f3653125b234c1a8b8b3f3ec944f448 2023-03-09 01:54:23,053 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/.tmp/family1/2f3653125b234c1a8b8b3f3ec944f448 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/2f3653125b234c1a8b8b3f3ec944f448 2023-03-09 01:54:23,057 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/2f3653125b234c1a8b8b3f3ec944f448, entries=2, sequenceid=9, filesize=5.0 K 2023-03-09 01:54:23,058 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 51e7af8c834c7e1949ec8753f22a28d7 in 840ms, sequenceid=9, compaction requested=false 2023-03-09 01:54:23,059 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 51e7af8c834c7e1949ec8753f22a28d7: 2023-03-09 01:54:23,059 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:23,059 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:54:23,059 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:54:23,059 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,059 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-09 01:54:23,060 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-09 01:54:23,062 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,062 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:23,062 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,063 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:23,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-09 01:54:23,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@614e0303[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:23,064 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-09 01:54:23,064 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,065 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,066 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-09 01:54:23,066 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-09 01:54:23,066 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,066 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:23,066 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:23,067 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,067 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-09 01:54:23,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:23,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:23,067 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:23,068 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-09 01:54:23,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:23,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:23,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:23,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:23,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-09 01:54:23,070 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-09 01:54:23,070 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:23,070 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:23,070 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,070 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:23,072 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,072 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:23,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:23,072 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:23,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:23,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:23,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:23,072 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:23,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,074 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,076 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,076 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:23,076 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,076 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-09 01:54:23,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:23,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:23,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-09 01:54:23,076 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:23,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:23,077 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:23,076 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:23,077 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,077 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:23,077 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:23,077 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:23,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:23,077 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,177 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:23,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:23,185 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-09 01:54:23,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-09 01:54:23,187 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-09 01:54:23,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:23,188 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327463188 2023-03-09 01:54:23,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:54:23,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-09 01:54:23,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-09 01:54:23,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x3dcdead5 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:54:23,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23f95127, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:54:23,194 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230309 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-09 01:54:24,023 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309838924c208b94725a8a9377235dd26ca to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230309838924c208b94725a8a9377235dd26ca 2023-03-09 01:54:24,024 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-09 01:54:24,037 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:54:24,039 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39830, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:54:24,046 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:54:24,048 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33712, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:54:24,057 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230309/family1/b55a2c7b16d046fc93cd71a39d676c6b first=Optional[k0] last=Optional[k1] 2023-03-09 01:54:24,058 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230309/family1/b55a2c7b16d046fc93cd71a39d676c6b}] 2023-03-09 01:54:24,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230309/family1/b55a2c7b16d046fc93cd71a39d676c6b for inclusion in 51e7af8c834c7e1949ec8753f22a28d7/family1 2023-03-09 01:54:24,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-09 01:54:24,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-09 01:54:24,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 51e7af8c834c7e1949ec8753f22a28d7: 2023-03-09 01:54:24,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230309/family1/b55a2c7b16d046fc93cd71a39d676c6b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testScannerOnBulkLoadRefHFiles__sr70hshfl3joupd0qdui9mss8bouc8ae227a5ji520bp6o6etm2ek6jrcg33bt9f/family1/b55a2c7b16d046fc93cd71a39d676c6b 2023-03-09 01:54:24,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testScannerOnBulkLoadRefHFiles__sr70hshfl3joupd0qdui9mss8bouc8ae227a5ji520bp6o6etm2ek6jrcg33bt9f/family1/b55a2c7b16d046fc93cd71a39d676c6b as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_ 2023-03-09 01:54:24,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testScannerOnBulkLoadRefHFiles__sr70hshfl3joupd0qdui9mss8bouc8ae227a5ji520bp6o6etm2ek6jrcg33bt9f/family1/b55a2c7b16d046fc93cd71a39d676c6b into 51e7af8c834c7e1949ec8753f22a28d7/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_ - updating store file list. 2023-03-09 01:54:24,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_ into 51e7af8c834c7e1949ec8753f22a28d7/family1 2023-03-09 01:54:24,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testScannerOnBulkLoadRefHFiles__sr70hshfl3joupd0qdui9mss8bouc8ae227a5ji520bp6o6etm2ek6jrcg33bt9f/family1/b55a2c7b16d046fc93cd71a39d676c6b into 51e7af8c834c7e1949ec8753f22a28d7/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_) 2023-03-09 01:54:24,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testScannerOnBulkLoadRefHFiles__sr70hshfl3joupd0qdui9mss8bouc8ae227a5ji520bp6o6etm2ek6jrcg33bt9f/family1/b55a2c7b16d046fc93cd71a39d676c6b 2023-03-09 01:54:24,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-09 01:54:24,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-09 01:54:24,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 51e7af8c834c7e1949ec8753f22a28d7/family1 is initiating minor compaction (all files) 2023-03-09 01:54:24,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1f33f468; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:24,076 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 51e7af8c834c7e1949ec8753f22a28d7/family1 in testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:54:24,076 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/870971919a7b469d8d48f115af9b9b51, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/2f3653125b234c1a8b8b3f3ec944f448, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/.tmp, totalSize=15.1 K 2023-03-09 01:54:24,077 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 870971919a7b469d8d48f115af9b9b51, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326861608 2023-03-09 01:54:24,077 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 2f3653125b234c1a8b8b3f3ec944f448, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326861608 2023-03-09 01:54:24,077 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326861608 2023-03-09 01:54:24,078 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-09 01:54:24,079 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:24,080 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030987d299ef52cd47b3a4cb8a16af01f6a2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023030987d299ef52cd47b3a4cb8a16af01f6a2 2023-03-09 01:54:24,081 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230309bed500ab33ff4303a8d509b9ee5ff054 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230309bed500ab33ff4303a8d509b9ee5ff054 2023-03-09 01:54:24,081 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-09 01:54:24,081 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230309' 2023-03-09 01:54:24,082 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230309 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-09 01:54:24,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-09 01:54:24,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:24,083 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-09 01:54:24,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 897 msec 2023-03-09 01:54:24,088 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 51e7af8c834c7e1949ec8753f22a28d7#family1#compaction#37 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:54:24,115 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=252 (was 255), OpenFileDescriptor=613 (was 608) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=221 (was 214) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=7995 (was 8589) 2023-03-09 01:54:24,130 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=252, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=221, ProcessCount=171, AvailableMemoryMB=7994 2023-03-09 01:54:24,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-09 01:54:24,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:24,135 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:54:24,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-09 01:54:24,136 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:54:24,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-09 01:54:24,137 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,138 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a empty. 2023-03-09 01:54:24,138 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,138 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-09 01:54:24,146 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-09 01:54:24,147 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => a4a8be322482491239e225079ce61b3a, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:54:24,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-09 01:54:24,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-09 01:54:24,558 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:24,559 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing a4a8be322482491239e225079ce61b3a, disabling compactions & flushes 2023-03-09 01:54:24,559 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:24,559 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:24,559 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. after waiting 0 ms 2023-03-09 01:54:24,559 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:24,559 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:24,559 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:24,561 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:54:24,562 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678326864562"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326864562"}]},"ts":"1678326864562"} 2023-03-09 01:54:24,564 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:54:24,565 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:54:24,565 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326864565"}]},"ts":"1678326864565"} 2023-03-09 01:54:24,566 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-09 01:54:24,569 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=a4a8be322482491239e225079ce61b3a, ASSIGN}] 2023-03-09 01:54:24,570 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=a4a8be322482491239e225079ce61b3a, ASSIGN 2023-03-09 01:54:24,571 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=a4a8be322482491239e225079ce61b3a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:54:24,722 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=a4a8be322482491239e225079ce61b3a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:24,722 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678326864722"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326864722"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326864722"}]},"ts":"1678326864722"} 2023-03-09 01:54:24,724 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure a4a8be322482491239e225079ce61b3a, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:24,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-09 01:54:24,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:24,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a4a8be322482491239e225079ce61b3a, NAME => 'testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:54:24,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:24,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,881 INFO [StoreOpener-a4a8be322482491239e225079ce61b3a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,882 DEBUG [StoreOpener-a4a8be322482491239e225079ce61b3a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1 2023-03-09 01:54:24,882 DEBUG [StoreOpener-a4a8be322482491239e225079ce61b3a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1 2023-03-09 01:54:24,883 INFO [StoreOpener-a4a8be322482491239e225079ce61b3a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4a8be322482491239e225079ce61b3a columnFamilyName family1 2023-03-09 01:54:24,884 INFO [StoreOpener-a4a8be322482491239e225079ce61b3a-1] regionserver.HStore(310): Store=a4a8be322482491239e225079ce61b3a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:24,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a4a8be322482491239e225079ce61b3a 2023-03-09 01:54:24,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:54:24,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a4a8be322482491239e225079ce61b3a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59279383, jitterRate=-0.11666835844516754}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:54:24,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:24,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a., pid=37, masterSystemTime=1678326864876 2023-03-09 01:54:24,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:24,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:24,894 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=a4a8be322482491239e225079ce61b3a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:24,894 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1678326864894"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326864894"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326864894"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326864894"}]},"ts":"1678326864894"} 2023-03-09 01:54:24,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-09 01:54:24,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure a4a8be322482491239e225079ce61b3a, server=jenkins-hbase4.apache.org,46511,1678326819757 in 171 msec 2023-03-09 01:54:24,899 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-09 01:54:24,899 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=a4a8be322482491239e225079ce61b3a, ASSIGN in 328 msec 2023-03-09 01:54:24,899 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:54:24,899 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326864899"}]},"ts":"1678326864899"} 2023-03-09 01:54:24,901 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-09 01:54:24,904 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:54:24,905 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 772 msec 2023-03-09 01:54:24,910 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/.tmp/family1/f130d048d3c449f4ba70876285b69268 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/f130d048d3c449f4ba70876285b69268 2023-03-09 01:54:24,915 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 51e7af8c834c7e1949ec8753f22a28d7/family1 of 51e7af8c834c7e1949ec8753f22a28d7 into f130d048d3c449f4ba70876285b69268(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:54:24,915 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 51e7af8c834c7e1949ec8753f22a28d7: 2023-03-09 01:54:24,915 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1678326864076; duration=0sec 2023-03-09 01:54:24,915 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:25,008 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/870971919a7b469d8d48f115af9b9b51, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/2f3653125b234c1a8b8b3f3ec944f448, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_] to archive 2023-03-09 01:54:25,010 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:25,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/870971919a7b469d8d48f115af9b9b51 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/870971919a7b469d8d48f115af9b9b51 2023-03-09 01:54:25,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/2f3653125b234c1a8b8b3f3ec944f448 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/2f3653125b234c1a8b8b3f3ec944f448 2023-03-09 01:54:25,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/family1/06fabce0127d4d4fbbb9edf08ab269ea_SeqId_12_ 2023-03-09 01:54:25,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-09 01:54:25,241 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-09 01:54:25,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:25,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,272 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:25,272 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:25,272 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:25,272 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:25,272 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,272 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:25,274 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:25,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:25,274 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,274 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:25,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:25,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,275 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,275 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:25,275 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:25,275 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:25,275 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:25,275 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:25,275 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:25,275 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:25,276 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-09 01:54:25,700 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b5770715e7ad4a10bc758bb552efc6d4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b5770715e7ad4a10bc758bb552efc6d4 2023-03-09 01:54:25,703 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/bb70ef30e2984e8cb90c1864aafb5fd9 2023-03-09 01:54:25,708 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/bb70ef30e2984e8cb90c1864aafb5fd9 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/bb70ef30e2984e8cb90c1864aafb5fd9 2023-03-09 01:54:25,713 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/bb70ef30e2984e8cb90c1864aafb5fd9, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-09 01:54:25,714 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 438ms, sequenceid=5, compaction requested=false 2023-03-09 01:54:25,714 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:25,714 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:25,715 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:25,715 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:25,715 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:25,715 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,715 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:25,715 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:25,716 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,716 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:25,717 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,717 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:25,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:25,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ee633ae[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:25,718 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:25,718 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,719 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,720 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:25,720 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:25,720 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:25,720 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,720 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:25,722 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,722 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:25,722 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:25,722 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:25,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:25,725 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:25,725 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:25,725 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:25,725 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,725 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:25,726 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,726 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:25,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:25,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,726 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:25,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:25,727 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,728 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:25,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:25,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,730 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:25,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:25,730 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:25,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:25,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:25,831 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:25,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:25,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:25,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:25,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,838 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:25,838 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:25,838 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:25,838 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:25,839 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,839 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,840 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,840 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:25,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:25,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:25,840 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:25,840 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:25,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:25,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,841 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,841 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:25,841 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:25,841 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:25,841 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:25,842 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:25,842 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:25,842 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:25,842 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:25,842 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:25,842 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-09 01:54:26,210 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:54:26,255 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280547e54b21f44138aca1d4faca2ab981 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511280547e54b21f44138aca1d4faca2ab981 2023-03-09 01:54:26,661 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/b485ffdcc9884ec2b4dbb374a674b749 2023-03-09 01:54:26,667 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/b485ffdcc9884ec2b4dbb374a674b749 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/b485ffdcc9884ec2b4dbb374a674b749 2023-03-09 01:54:26,672 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/b485ffdcc9884ec2b4dbb374a674b749, entries=1, sequenceid=9, filesize=5.0 K 2023-03-09 01:54:26,673 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 831ms, sequenceid=9, compaction requested=false 2023-03-09 01:54:26,673 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:26,673 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:26,673 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:26,673 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:26,673 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,673 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:26,673 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:26,676 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,676 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:26,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:26,676 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,676 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:26,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:26,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:26,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:26,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:26,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@167bf6f6[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:26,677 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:26,677 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,680 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:26,681 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,681 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:26,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:26,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:26,682 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,682 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:26,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:26,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:26,682 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:26,682 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:26,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:26,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:26,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:26,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:26,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:26,684 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:26,684 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:26,684 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:26,685 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,685 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:26,687 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,687 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:26,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,687 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:26,687 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,689 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,691 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:26,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:26,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:26,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:26,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:26,691 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:26,791 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:26,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:26,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:26,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:26,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,798 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:26,798 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:26,798 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:26,798 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:26,798 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,799 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,800 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,800 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:26,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:26,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:26,800 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:26,800 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:26,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,801 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:26,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,801 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:26,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:26,802 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:26,802 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:26,802 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:27,214 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201b86ab0637d37478d9618f911d0ad9573 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201b86ab0637d37478d9618f911d0ad9573 2023-03-09 01:54:27,621 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/10162a912832420186132058cf297f13 2023-03-09 01:54:27,627 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/10162a912832420186132058cf297f13 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/10162a912832420186132058cf297f13 2023-03-09 01:54:27,631 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/10162a912832420186132058cf297f13, entries=1, sequenceid=13, filesize=4.9 K 2023-03-09 01:54:27,632 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 830ms, sequenceid=13, compaction requested=true 2023-03-09 01:54:27,632 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:27,632 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:27,632 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:27,632 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:27,632 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,632 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:27,632 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:27,634 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,634 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:27,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:27,634 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,634 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:27,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:27,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:27,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:27,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:27,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74309187[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:27,636 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:27,636 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,638 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,638 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:27,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:27,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:27,639 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,639 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:27,640 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,640 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:27,640 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:27,641 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:27,643 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:27,643 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:27,643 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:27,643 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,643 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:27,645 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,645 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,645 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:27,645 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:27,646 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:27,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:27,649 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:27,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:27,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:27,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:27,649 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:27,650 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:27,650 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:27,750 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:27,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:27,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:27,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:27,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,757 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:27,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:27,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:27,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:27,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:27,760 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:27,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:27,760 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:27,760 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:27,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:27,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:27,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:27,762 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:27,762 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:27,762 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:27,762 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:27,762 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:27,762 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:27,762 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:27,762 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:28,176 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205dc6de1920e0247a4a3cdccffbae79512 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205dc6de1920e0247a4a3cdccffbae79512 2023-03-09 01:54:28,582 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/8bf221f7b12141ec9c50979921c850ef 2023-03-09 01:54:28,587 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/8bf221f7b12141ec9c50979921c850ef as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8bf221f7b12141ec9c50979921c850ef 2023-03-09 01:54:28,591 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8bf221f7b12141ec9c50979921c850ef, entries=1, sequenceid=17, filesize=4.9 K 2023-03-09 01:54:28,592 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 830ms, sequenceid=17, compaction requested=true 2023-03-09 01:54:28,592 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:28,592 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:28,592 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:28,592 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:28,592 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,592 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:28,592 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:28,594 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,594 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:28,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:28,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:28,594 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,594 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:28,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:28,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:28,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:28,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65f7c219[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:28,595 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:28,595 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,597 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,597 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,597 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:28,597 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:28,597 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:28,597 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:28,598 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,598 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:28,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:28,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:28,599 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:28,599 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:28,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:28,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:28,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:28,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:28,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:28,601 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:28,601 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:28,601 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:28,601 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,601 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:28,603 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,603 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:28,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:28,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,603 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:28,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:28,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:28,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:28,604 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:28,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,605 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:28,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,607 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,607 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,607 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:28,607 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:28,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:28,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:28,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:28,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:28,607 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:28,607 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,607 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:28,608 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,608 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,608 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,608 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:28,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:28,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:28,608 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:28,708 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:28,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:28,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:28,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:28,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,715 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:28,715 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:28,715 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:28,715 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:28,715 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,715 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,717 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,717 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:28,717 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:28,717 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:28,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:28,718 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:28,718 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:28,718 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:28,718 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:28,718 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:28,718 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:28,718 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:28,718 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:29,131 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228be1b2814b13642a5b919fea1fd24164b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228be1b2814b13642a5b919fea1fd24164b 2023-03-09 01:54:29,537 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/9fb92745ee854b81ba443a4c4cfe63e7 2023-03-09 01:54:29,542 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/9fb92745ee854b81ba443a4c4cfe63e7 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/9fb92745ee854b81ba443a4c4cfe63e7 2023-03-09 01:54:29,546 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/9fb92745ee854b81ba443a4c4cfe63e7, entries=1, sequenceid=21, filesize=4.9 K 2023-03-09 01:54:29,547 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 829ms, sequenceid=21, compaction requested=true 2023-03-09 01:54:29,547 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:29,547 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:29,556 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,556 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:29,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:29,557 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,557 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:29,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:29,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a83f64a[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:29,558 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:29,558 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,559 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,560 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:29,560 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:29,560 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:29,560 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,560 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:29,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:29,561 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:29,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:29,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:29,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:29,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:29,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:29,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:29,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:29,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:29,563 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:29,563 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:29,563 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:29,563 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,563 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:29,565 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,565 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,565 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:29,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:29,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:29,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:29,566 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:29,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,567 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:29,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,570 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,570 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,570 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:29,570 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:29,570 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,570 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:29,570 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:29,570 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:29,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:29,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:29,570 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,571 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,571 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:29,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:29,571 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:29,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:29,671 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:29,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:29,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:29,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:29,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,678 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:29,678 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:29,678 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:29,678 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:29,678 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,678 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,680 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:29,680 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:29,680 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:29,680 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,681 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,681 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:29,681 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:29,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:29,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:29,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:29,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:29,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:29,681 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:29,681 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:29,682 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:30,094 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231e829dc4db5b349e29b89f7f5e8e93b8e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231e829dc4db5b349e29b89f7f5e8e93b8e 2023-03-09 01:54:30,500 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/3e890460adea4431a3b1a576d5e579e9 2023-03-09 01:54:30,505 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/3e890460adea4431a3b1a576d5e579e9 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3e890460adea4431a3b1a576d5e579e9 2023-03-09 01:54:30,509 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3e890460adea4431a3b1a576d5e579e9, entries=1, sequenceid=25, filesize=4.9 K 2023-03-09 01:54:30,510 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 828ms, sequenceid=25, compaction requested=true 2023-03-09 01:54:30,510 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:30,510 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:30,510 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:30,510 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:30,510 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,510 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:30,510 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:30,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,512 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:30,512 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,512 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:30,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:30,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@230671aa[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:30,513 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:30,514 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,515 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,515 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:30,515 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:30,515 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:30,515 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,515 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:30,518 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,518 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:30,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:30,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:30,518 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:30,518 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:30,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:30,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:30,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:30,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:30,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:30,520 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:30,520 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:30,520 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:30,520 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,520 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:30,522 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,522 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:30,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:30,522 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:30,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:30,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:30,522 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:30,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:30,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,524 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:30,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:30,526 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:30,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:30,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:30,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:30,526 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:30,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:30,627 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:30,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:30,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:30,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:30,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,633 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:30,633 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:30,633 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:30,633 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:30,634 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,634 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,635 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,635 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:30,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:30,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:30,635 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:30,635 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:30,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:30,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:30,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:30,636 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:30,636 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:30,637 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:30,637 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:30,637 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:30,637 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:30,637 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:30,637 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:31,049 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010182da5db68a6f4426a60389aaee3b34ee to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010182da5db68a6f4426a60389aaee3b34ee 2023-03-09 01:54:31,455 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/ffb3e746419f45ab872fe03beb65e64d 2023-03-09 01:54:31,459 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/ffb3e746419f45ab872fe03beb65e64d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ffb3e746419f45ab872fe03beb65e64d 2023-03-09 01:54:31,463 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ffb3e746419f45ab872fe03beb65e64d, entries=1, sequenceid=29, filesize=4.9 K 2023-03-09 01:54:31,464 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 827ms, sequenceid=29, compaction requested=true 2023-03-09 01:54:31,464 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:31,464 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:31,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:31,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:31,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:31,464 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:31,466 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,466 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:31,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:31,467 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,467 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:31,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:31,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:31,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:31,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:31,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53ca60e4[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:31,468 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:31,468 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,469 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,470 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:31,470 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:31,470 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:31,470 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,470 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:31,472 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:31,472 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,472 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:31,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:31,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:31,472 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:31,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:31,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:31,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:31,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:31,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:31,474 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:31,474 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:31,474 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:31,474 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,474 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:31,480 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,480 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,480 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:31,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:31,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:31,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:31,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:31,481 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,482 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:31,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:31,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:31,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:31,485 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,485 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,486 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:31,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:31,585 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:31,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:31,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:31,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:31,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,592 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:31,592 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:31,592 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:31,592 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:31,593 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,593 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,602 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,602 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:31,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:31,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:31,602 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:31,602 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:31,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,603 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:31,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,603 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:31,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:31,603 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:31,603 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:31,603 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:31,603 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:31,603 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:31,603 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:31,603 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:31,604 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:32,016 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c6570d3fb12e442a861dfeb2d0ba861f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103c6570d3fb12e442a861dfeb2d0ba861f 2023-03-09 01:54:32,422 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/ad61cc79f6eb404798a1176e606beca1 2023-03-09 01:54:32,427 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/ad61cc79f6eb404798a1176e606beca1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ad61cc79f6eb404798a1176e606beca1 2023-03-09 01:54:32,431 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ad61cc79f6eb404798a1176e606beca1, entries=1, sequenceid=33, filesize=4.9 K 2023-03-09 01:54:32,432 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 828ms, sequenceid=33, compaction requested=true 2023-03-09 01:54:32,432 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:32,432 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:32,434 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,434 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:32,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:32,434 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,434 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:32,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:32,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:32,436 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:32,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8f6878e[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:32,436 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,438 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,438 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:32,438 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:32,438 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:32,438 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,439 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:32,440 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,440 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:32,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:32,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:32,440 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:32,440 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:32,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:32,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:32,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:32,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:32,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:32,442 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:32,443 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:32,443 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:32,443 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,443 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:32,444 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,444 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:32,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:32,444 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:32,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:32,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:32,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:32,445 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:32,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,446 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:32,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,448 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,448 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:32,448 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:32,449 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:32,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:32,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:32,449 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,449 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:32,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:32,449 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,449 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,449 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,449 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:32,449 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:32,449 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:32,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:32,549 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:32,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:32,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:32,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:32,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,556 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:32,556 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:32,556 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:32,556 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:32,557 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,557 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,559 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:32,559 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:32,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:32,560 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:32,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,560 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:32,560 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:32,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,560 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,561 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:32,561 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:32,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:32,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:32,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:32,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:32,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:32,561 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:32,561 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:32,561 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:32,974 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907855e2281ecbf4ba69a54cc209e4ec9b5 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907855e2281ecbf4ba69a54cc209e4ec9b5 2023-03-09 01:54:33,379 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/8a25642440f541888104a576cdbeb15c 2023-03-09 01:54:33,385 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/8a25642440f541888104a576cdbeb15c as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8a25642440f541888104a576cdbeb15c 2023-03-09 01:54:33,389 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8a25642440f541888104a576cdbeb15c, entries=1, sequenceid=37, filesize=4.9 K 2023-03-09 01:54:33,390 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 829ms, sequenceid=37, compaction requested=true 2023-03-09 01:54:33,390 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:33,390 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:33,390 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:33,390 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:33,391 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,391 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:33,391 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:33,392 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,393 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:33,393 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,393 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:33,394 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7db2017c[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:33,394 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,395 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,395 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,396 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:33,395 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:33,396 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:33,396 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:33,397 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:33,397 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,397 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:33,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:33,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:33,397 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:33,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:33,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:33,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:33,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:33,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:33,399 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:33,399 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:33,399 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:33,399 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,399 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:33,401 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,401 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,402 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:33,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:33,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:33,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:33,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:33,402 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:33,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:33,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,403 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:33,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,405 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,405 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:33,405 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,405 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:33,405 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:33,405 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-09 01:54:33,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:33,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:33,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:33,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:33,405 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,406 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,406 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,406 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,406 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,406 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:33,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:33,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:33,406 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:33,506 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:33,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:33,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:33,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:33,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,513 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:33,513 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:33,513 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-09 01:54:33,513 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-09 01:54:33,514 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,514 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,515 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:33,515 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:33,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:33,515 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:33,515 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:33,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:33,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-09 01:54:33,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:33,516 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-09 01:54:33,516 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:33,516 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-09 01:54:33,517 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:33,517 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:33,517 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:33,517 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. started... 2023-03-09 01:54:33,517 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a4a8be322482491239e225079ce61b3a 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:33,930 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e0231f7efe5c4d12a516c9fe720e8ed9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120e0231f7efe5c4d12a516c9fe720e8ed9 2023-03-09 01:54:34,335 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/8aa27f918b874ee09112af94c76c9ef8 2023-03-09 01:54:34,340 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/8aa27f918b874ee09112af94c76c9ef8 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8aa27f918b874ee09112af94c76c9ef8 2023-03-09 01:54:34,344 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8aa27f918b874ee09112af94c76c9ef8, entries=1, sequenceid=41, filesize=4.9 K 2023-03-09 01:54:34,345 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a4a8be322482491239e225079ce61b3a in 828ms, sequenceid=41, compaction requested=true 2023-03-09 01:54:34,345 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:34,345 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:34,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:34,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:34,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-09 01:54:34,345 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-09 01:54:34,348 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,348 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:34,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:34,348 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,348 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:34,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:34,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:34,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:34,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-09 01:54:34,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d29b441[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:34,349 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-09 01:54:34,349 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,352 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,352 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-09 01:54:34,352 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-09 01:54:34,352 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:34,352 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,352 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:34,354 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,354 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-09 01:54:34,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:34,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:34,354 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:34,354 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-09 01:54:34,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:34,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:34,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:34,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:34,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:54:34,356 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-09 01:54:34,356 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:34,356 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:34,356 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,356 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:34,357 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,357 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:34,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:34,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,357 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:34,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:34,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:34,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:34,358 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:34,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,359 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:34,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,364 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,364 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:34,364 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-09 01:54:34,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:34,364 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:34,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:34,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:34,364 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:34,364 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,365 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:34,365 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,365 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:34,365 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:34,365 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:34,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:34,365 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,465 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:34,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:34,467 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-09 01:54:34,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:34,470 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:34,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:34,471 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327474471 2023-03-09 01:54:34,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:54:34,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-09 01:54:34,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:34,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x201ff8b0 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:54:34,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4986f5fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:54:34,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,316 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bf4bde5ec39f4cc4a50cea7be857c4c9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103bf4bde5ec39f4cc4a50cea7be857c4c9 2023-03-09 01:54:35,317 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:35,329 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:54:35,330 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59832, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:54:35,337 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:54:35,338 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46010, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:54:35,348 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/70e3b1bf06a2492dabbf4ae8437dc100 first=Optional[r3] last=Optional[r6] 2023-03-09 01:54:35,349 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/70e3b1bf06a2492dabbf4ae8437dc100}] 2023-03-09 01:54:35,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/70e3b1bf06a2492dabbf4ae8437dc100 for inclusion in a4a8be322482491239e225079ce61b3a/family1 2023-03-09 01:54:35,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-09 01:54:35,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:35,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:35,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/70e3b1bf06a2492dabbf4ae8437dc100 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__nevta7r5v5nvq1faoocam4k62tvlhnfrmhsl7q520aren36fj8t3tiuof7is695a/family1/70e3b1bf06a2492dabbf4ae8437dc100 2023-03-09 01:54:35,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__nevta7r5v5nvq1faoocam4k62tvlhnfrmhsl7q520aren36fj8t3tiuof7is695a/family1/70e3b1bf06a2492dabbf4ae8437dc100 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/d70e20db25c540118332948bd1e1ab00_SeqId_44_ 2023-03-09 01:54:35,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__nevta7r5v5nvq1faoocam4k62tvlhnfrmhsl7q520aren36fj8t3tiuof7is695a/family1/70e3b1bf06a2492dabbf4ae8437dc100 into a4a8be322482491239e225079ce61b3a/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/d70e20db25c540118332948bd1e1ab00_SeqId_44_ - updating store file list. 2023-03-09 01:54:35,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/d70e20db25c540118332948bd1e1ab00_SeqId_44_ into a4a8be322482491239e225079ce61b3a/family1 2023-03-09 01:54:35,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__nevta7r5v5nvq1faoocam4k62tvlhnfrmhsl7q520aren36fj8t3tiuof7is695a/family1/70e3b1bf06a2492dabbf4ae8437dc100 into a4a8be322482491239e225079ce61b3a/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/d70e20db25c540118332948bd1e1ab00_SeqId_44_) 2023-03-09 01:54:35,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__nevta7r5v5nvq1faoocam4k62tvlhnfrmhsl7q520aren36fj8t3tiuof7is695a/family1/70e3b1bf06a2492dabbf4ae8437dc100 2023-03-09 01:54:35,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-09 01:54:35,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-09 01:54:35,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): a4a8be322482491239e225079ce61b3a/family1 is initiating minor compaction 2023-03-09 01:54:35,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7a054fb1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:35,366 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a4a8be322482491239e225079ce61b3a/family1 in testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:35,366 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/b485ffdcc9884ec2b4dbb374a674b749, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/10162a912832420186132058cf297f13, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8bf221f7b12141ec9c50979921c850ef, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/9fb92745ee854b81ba443a4c4cfe63e7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3e890460adea4431a3b1a576d5e579e9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ffb3e746419f45ab872fe03beb65e64d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ad61cc79f6eb404798a1176e606beca1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8a25642440f541888104a576cdbeb15c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8aa27f918b874ee09112af94c76c9ef8, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/d70e20db25c540118332948bd1e1ab00_SeqId_44_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp, totalSize=50.0 K 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting b485ffdcc9884ec2b4dbb374a674b749, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 10162a912832420186132058cf297f13, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 8bf221f7b12141ec9c50979921c850ef, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 9fb92745ee854b81ba443a4c4cfe63e7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 3e890460adea4431a3b1a576d5e579e9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting ffb3e746419f45ab872fe03beb65e64d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting ad61cc79f6eb404798a1176e606beca1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 8a25642440f541888104a576cdbeb15c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 8aa27f918b874ee09112af94c76c9ef8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-09 01:54:35,366 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting d70e20db25c540118332948bd1e1ab00_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-09 01:54:35,367 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:35,368 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:35,370 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228be1b2814b13642a5b919fea1fd24164b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228be1b2814b13642a5b919fea1fd24164b 2023-03-09 01:54:35,371 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231e829dc4db5b349e29b89f7f5e8e93b8e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231e829dc4db5b349e29b89f7f5e8e93b8e 2023-03-09 01:54:35,372 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010182da5db68a6f4426a60389aaee3b34ee to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010182da5db68a6f4426a60389aaee3b34ee 2023-03-09 01:54:35,374 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103c6570d3fb12e442a861dfeb2d0ba861f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103c6570d3fb12e442a861dfeb2d0ba861f 2023-03-09 01:54:35,374 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-09 01:54:35,374 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-09 01:54:35,374 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,374 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,374 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,409 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): a4a8be322482491239e225079ce61b3a#family1#compaction#48 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:54:35,847 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/5a01585b8824416c813e4a49b1754e96 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/5a01585b8824416c813e4a49b1754e96 2023-03-09 01:54:35,851 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120589af18a8008644fc853f7ea5f42b42eb to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120589af18a8008644fc853f7ea5f42b42eb 2023-03-09 01:54:35,852 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:35,852 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a4a8be322482491239e225079ce61b3a/family1 of a4a8be322482491239e225079ce61b3a into 5a01585b8824416c813e4a49b1754e96(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:54:35,852 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:35,852 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1678326875366; duration=0sec 2023-03-09 01:54:35,852 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:35,881 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/594f9620a8fa408e8ec987123f40f0ff first=Optional[r00] last=Optional[r2] 2023-03-09 01:54:35,882 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/594f9620a8fa408e8ec987123f40f0ff}] 2023-03-09 01:54:35,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/594f9620a8fa408e8ec987123f40f0ff for inclusion in a4a8be322482491239e225079ce61b3a/family1 2023-03-09 01:54:35,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-09 01:54:35,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:35,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:35,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/594f9620a8fa408e8ec987123f40f0ff to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qbls7ffo07hv8qj7puj6qr8bunmdbm4dhgc3lp0571ed091fj77hplf0m6i2qmmn/family1/594f9620a8fa408e8ec987123f40f0ff 2023-03-09 01:54:35,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qbls7ffo07hv8qj7puj6qr8bunmdbm4dhgc3lp0571ed091fj77hplf0m6i2qmmn/family1/594f9620a8fa408e8ec987123f40f0ff as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3cebb4811e604f6a93ecb7999b27b163_SeqId_47_ 2023-03-09 01:54:35,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qbls7ffo07hv8qj7puj6qr8bunmdbm4dhgc3lp0571ed091fj77hplf0m6i2qmmn/family1/594f9620a8fa408e8ec987123f40f0ff into a4a8be322482491239e225079ce61b3a/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3cebb4811e604f6a93ecb7999b27b163_SeqId_47_ - updating store file list. 2023-03-09 01:54:35,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3cebb4811e604f6a93ecb7999b27b163_SeqId_47_ into a4a8be322482491239e225079ce61b3a/family1 2023-03-09 01:54:35,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qbls7ffo07hv8qj7puj6qr8bunmdbm4dhgc3lp0571ed091fj77hplf0m6i2qmmn/family1/594f9620a8fa408e8ec987123f40f0ff into a4a8be322482491239e225079ce61b3a/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3cebb4811e604f6a93ecb7999b27b163_SeqId_47_) 2023-03-09 01:54:35,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qbls7ffo07hv8qj7puj6qr8bunmdbm4dhgc3lp0571ed091fj77hplf0m6i2qmmn/family1/594f9620a8fa408e8ec987123f40f0ff 2023-03-09 01:54:35,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-09 01:54:35,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-09 01:54:35,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): a4a8be322482491239e225079ce61b3a/family1 is initiating minor compaction (all files) 2023-03-09 01:54:35,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4ee5ca7f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:35,898 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a4a8be322482491239e225079ce61b3a/family1 in testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:54:35,898 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/bb70ef30e2984e8cb90c1864aafb5fd9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/5a01585b8824416c813e4a49b1754e96, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3cebb4811e604f6a93ecb7999b27b163_SeqId_47_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp, totalSize=337.0 K 2023-03-09 01:54:35,899 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting bb70ef30e2984e8cb90c1864aafb5fd9, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843359 2023-03-09 01:54:35,899 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 5a01585b8824416c813e4a49b1754e96, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243359 2023-03-09 01:54:35,899 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 3cebb4811e604f6a93ecb7999b27b163_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878843359 2023-03-09 01:54:35,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:35,901 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:35,902 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b5770715e7ad4a10bc758bb552efc6d4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b5770715e7ad4a10bc758bb552efc6d4 2023-03-09 01:54:35,904 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201b86ab0637d37478d9618f911d0ad9573 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201b86ab0637d37478d9618f911d0ad9573 2023-03-09 01:54:35,905 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205dc6de1920e0247a4a3cdccffbae79512 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205dc6de1920e0247a4a3cdccffbae79512 2023-03-09 01:54:35,905 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-09 01:54:35,905 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-09 01:54:35,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-09 01:54:35,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-09 01:54:35,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:35,907 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:54:35,917 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.4390 sec 2023-03-09 01:54:35,919 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/b485ffdcc9884ec2b4dbb374a674b749, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/10162a912832420186132058cf297f13, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8bf221f7b12141ec9c50979921c850ef, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/9fb92745ee854b81ba443a4c4cfe63e7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3e890460adea4431a3b1a576d5e579e9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ffb3e746419f45ab872fe03beb65e64d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ad61cc79f6eb404798a1176e606beca1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8a25642440f541888104a576cdbeb15c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8aa27f918b874ee09112af94c76c9ef8, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/d70e20db25c540118332948bd1e1ab00_SeqId_44_] to archive 2023-03-09 01:54:35,920 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:35,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/b485ffdcc9884ec2b4dbb374a674b749 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/b485ffdcc9884ec2b4dbb374a674b749 2023-03-09 01:54:35,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/10162a912832420186132058cf297f13 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/10162a912832420186132058cf297f13 2023-03-09 01:54:35,925 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8bf221f7b12141ec9c50979921c850ef to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8bf221f7b12141ec9c50979921c850ef 2023-03-09 01:54:35,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/9fb92745ee854b81ba443a4c4cfe63e7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/9fb92745ee854b81ba443a4c4cfe63e7 2023-03-09 01:54:35,927 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): a4a8be322482491239e225079ce61b3a#family1#compaction#49 average throughput is 14.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:54:35,928 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3e890460adea4431a3b1a576d5e579e9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3e890460adea4431a3b1a576d5e579e9 2023-03-09 01:54:35,929 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ffb3e746419f45ab872fe03beb65e64d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ffb3e746419f45ab872fe03beb65e64d 2023-03-09 01:54:35,931 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ad61cc79f6eb404798a1176e606beca1 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/ad61cc79f6eb404798a1176e606beca1 2023-03-09 01:54:35,932 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8a25642440f541888104a576cdbeb15c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8a25642440f541888104a576cdbeb15c 2023-03-09 01:54:35,933 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8aa27f918b874ee09112af94c76c9ef8 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/8aa27f918b874ee09112af94c76c9ef8 2023-03-09 01:54:35,935 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/d70e20db25c540118332948bd1e1ab00_SeqId_44_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/d70e20db25c540118332948bd1e1ab00_SeqId_44_ 2023-03-09 01:54:36,022 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=264 (was 252) - Thread LEAK? -, OpenFileDescriptor=628 (was 613) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 221), ProcessCount=171 (was 171), AvailableMemoryMB=8137 (was 7994) - AvailableMemoryMB LEAK? - 2023-03-09 01:54:36,035 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=264, OpenFileDescriptor=628, MaxFileDescriptor=60000, SystemLoadAverage=202, ProcessCount=171, AvailableMemoryMB=8137 2023-03-09 01:54:36,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-09 01:54:36,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:36,039 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:54:36,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-09 01:54:36,040 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:54:36,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-09 01:54:36,041 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,042 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 empty. 2023-03-09 01:54:36,042 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,042 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-09 01:54:36,050 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-09 01:54:36,051 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3a92a0ebc17a6fe227772466246c6162, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:54:36,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-09 01:54:36,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-09 01:54:36,456 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:36,456 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 3a92a0ebc17a6fe227772466246c6162, disabling compactions & flushes 2023-03-09 01:54:36,456 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:36,456 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:36,456 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. after waiting 0 ms 2023-03-09 01:54:36,456 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:36,456 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:36,456 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:36,459 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:54:36,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326876459"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326876459"}]},"ts":"1678326876459"} 2023-03-09 01:54:36,461 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:54:36,462 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:54:36,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326876462"}]},"ts":"1678326876462"} 2023-03-09 01:54:36,463 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-09 01:54:36,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, ASSIGN}] 2023-03-09 01:54:36,468 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, ASSIGN 2023-03-09 01:54:36,468 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:54:36,619 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:36,620 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326876619"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326876619"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326876619"}]},"ts":"1678326876619"} 2023-03-09 01:54:36,622 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:36,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-09 01:54:36,755 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/.tmp/family1/28070a8a58cb46d0a618c6d7c5280062 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/28070a8a58cb46d0a618c6d7c5280062 2023-03-09 01:54:36,760 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a4a8be322482491239e225079ce61b3a/family1 of a4a8be322482491239e225079ce61b3a into 28070a8a58cb46d0a618c6d7c5280062(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:54:36,760 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:54:36,760 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1678326875898; duration=0sec 2023-03-09 01:54:36,760 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:36,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:36,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3a92a0ebc17a6fe227772466246c6162, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:54:36,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:36,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,778 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,779 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:36,779 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:36,779 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3a92a0ebc17a6fe227772466246c6162 columnFamilyName family1 2023-03-09 01:54:36,781 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(310): Store=3a92a0ebc17a6fe227772466246c6162/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:36,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:36,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:54:36,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3a92a0ebc17a6fe227772466246c6162; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73223583, jitterRate=0.09111641347408295}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:54:36,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:36,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162., pid=41, masterSystemTime=1678326876774 2023-03-09 01:54:36,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:36,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:36,789 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:36,789 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326876789"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326876789"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326876789"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326876789"}]},"ts":"1678326876789"} 2023-03-09 01:54:36,792 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-09 01:54:36,792 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757 in 168 msec 2023-03-09 01:54:36,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-09 01:54:36,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, ASSIGN in 326 msec 2023-03-09 01:54:36,794 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:54:36,794 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326876794"}]},"ts":"1678326876794"} 2023-03-09 01:54:36,795 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-09 01:54:36,798 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:54:36,799 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 761 msec 2023-03-09 01:54:36,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/bb70ef30e2984e8cb90c1864aafb5fd9, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/5a01585b8824416c813e4a49b1754e96, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3cebb4811e604f6a93ecb7999b27b163_SeqId_47_] to archive 2023-03-09 01:54:36,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:36,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/bb70ef30e2984e8cb90c1864aafb5fd9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/bb70ef30e2984e8cb90c1864aafb5fd9 2023-03-09 01:54:36,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/5a01585b8824416c813e4a49b1754e96 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/5a01585b8824416c813e4a49b1754e96 2023-03-09 01:54:36,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3cebb4811e604f6a93ecb7999b27b163_SeqId_47_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/family1/3cebb4811e604f6a93ecb7999b27b163_SeqId_47_ 2023-03-09 01:54:37,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-09 01:54:37,145 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-09 01:54:37,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:37,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:37,174 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:37,174 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:37,174 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:37,174 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:37,174 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:37,175 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:37,176 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:37,176 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:37,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:37,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:37,177 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:37,177 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:37,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:37,177 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:37,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:37,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:37,178 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:37,178 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:37,178 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:37,178 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:37,178 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:37,178 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:37,178 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:37,178 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:37,179 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:37,179 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-09 01:54:37,607 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130eb7be2ff6a1a46e7a42a2e50d692cd95 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb7be2ff6a1a46e7a42a2e50d692cd95 2023-03-09 01:54:38,011 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/55e78b03425a488086bd6b1f021a582a 2023-03-09 01:54:38,017 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/55e78b03425a488086bd6b1f021a582a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/55e78b03425a488086bd6b1f021a582a 2023-03-09 01:54:38,021 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/55e78b03425a488086bd6b1f021a582a, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-09 01:54:38,022 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 843ms, sequenceid=5, compaction requested=false 2023-03-09 01:54:38,022 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:38,023 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:38,023 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:38,023 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:38,025 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,025 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:38,025 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:38,025 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:38,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:38,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:38,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6de42440[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:38,026 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:38,026 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,027 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,027 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:38,027 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:38,028 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:38,028 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,028 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:38,030 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,030 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:38,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,030 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:38,031 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:38,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:38,033 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:38,033 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:38,033 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:38,033 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:38,033 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,033 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:38,034 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,034 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,034 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:38,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:38,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:38,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:38,035 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:38,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:38,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:38,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,036 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:38,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:38,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:38,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:38,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-09 01:54:38,038 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:38,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:38,138 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:38,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:38,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:38,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:38,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,145 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:38,146 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:38,146 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:38,146 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:38,146 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,146 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,147 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,147 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:38,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:38,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:38,148 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,148 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:38,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:38,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:38,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:38,149 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:38,149 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:38,149 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:38,150 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-09 01:54:38,562 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128399d00cb74b344859fc713c01f61d25f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128399d00cb74b344859fc713c01f61d25f 2023-03-09 01:54:38,969 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/4837b65730e24c8e8fc8eafd65a8a0ce 2023-03-09 01:54:38,974 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/4837b65730e24c8e8fc8eafd65a8a0ce as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4837b65730e24c8e8fc8eafd65a8a0ce 2023-03-09 01:54:38,978 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4837b65730e24c8e8fc8eafd65a8a0ce, entries=1, sequenceid=9, filesize=5.0 K 2023-03-09 01:54:38,979 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 829ms, sequenceid=9, compaction requested=false 2023-03-09 01:54:38,979 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:38,979 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:38,979 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:38,980 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:38,980 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,980 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:38,980 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:38,982 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,982 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:38,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:38,983 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:38,983 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:38,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:38,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:38,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:38,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51120485[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:38,984 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:38,984 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,985 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,986 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:38,986 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:38,986 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:38,986 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,986 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:38,987 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,987 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:38,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:38,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:38,987 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:38,988 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:38,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:38,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:38,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:38,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:38,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:38,990 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:38,990 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:38,990 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:38,990 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,990 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:38,991 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,991 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:38,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:38,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,991 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:38,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:38,992 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:38,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:38,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:38,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,993 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:38,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:38,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:38,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,002 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:39,002 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:39,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:39,002 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:39,002 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:39,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:39,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:39,002 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,003 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,003 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,003 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:39,003 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:39,003 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:39,003 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:39,003 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,103 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:39,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:39,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:39,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:39,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,110 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:39,110 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:39,110 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:39,110 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:39,110 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,110 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,112 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:39,112 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:39,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:39,112 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,112 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:39,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:39,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,113 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:39,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,114 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:39,114 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:39,114 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:39,114 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:39,114 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:39,114 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:39,114 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:39,114 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:39,527 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e331fd7f3a9b4921af959b3deaf081ba to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201e331fd7f3a9b4921af959b3deaf081ba 2023-03-09 01:54:39,933 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/4c2ae58284264883839d0de8aa5e6d01 2023-03-09 01:54:39,938 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/4c2ae58284264883839d0de8aa5e6d01 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4c2ae58284264883839d0de8aa5e6d01 2023-03-09 01:54:39,942 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4c2ae58284264883839d0de8aa5e6d01, entries=1, sequenceid=13, filesize=5.0 K 2023-03-09 01:54:39,942 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 828ms, sequenceid=13, compaction requested=true 2023-03-09 01:54:39,942 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:39,942 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:39,943 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:39,943 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:39,943 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,943 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:39,943 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:39,944 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:39,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:39,944 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:39,945 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,945 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:39,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:39,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:39,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:39,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cc6463a[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:39,946 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:39,946 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,948 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,948 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:39,948 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:39,948 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,948 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:39,948 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:39,950 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,950 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:39,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:39,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:39,950 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:39,950 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:39,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:39,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:39,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:39,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:39,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:39,952 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:39,952 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:39,952 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:39,952 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,952 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:39,954 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,954 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:39,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:39,954 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:39,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,955 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:39,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:39,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:39,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:39,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,956 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:39,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,958 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:39,958 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:39,958 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,958 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:39,958 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:39,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:39,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:39,958 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:39,958 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:39,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-09 01:54:39,959 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,959 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,959 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:39,959 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:39,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:39,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:39,959 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:40,059 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:40,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:40,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:40,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:40,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,066 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:40,066 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:40,066 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:40,066 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:40,066 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,066 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,068 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:40,068 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:40,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:40,068 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,068 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:40,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,069 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:40,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:40,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,069 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:40,069 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:40,069 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:40,070 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:40,070 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:40,070 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:40,070 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:40,070 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:40,482 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e691d91b1f76408f89ade10976166129 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205e691d91b1f76408f89ade10976166129 2023-03-09 01:54:40,888 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/873e5dcff97244438f99ef299831ac00 2023-03-09 01:54:40,893 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/873e5dcff97244438f99ef299831ac00 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/873e5dcff97244438f99ef299831ac00 2023-03-09 01:54:40,896 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/873e5dcff97244438f99ef299831ac00, entries=1, sequenceid=17, filesize=5.0 K 2023-03-09 01:54:40,897 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 827ms, sequenceid=17, compaction requested=true 2023-03-09 01:54:40,897 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:40,897 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:40,900 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,900 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:40,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:40,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:40,900 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,900 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:40,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:40,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ced665f[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:40,902 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:40,902 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,903 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,903 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:40,903 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:40,903 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:40,903 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,903 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:40,904 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:40,904 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,904 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:40,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:40,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:40,904 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:40,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:40,906 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:40,906 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:40,907 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:40,907 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,907 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:40,908 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,908 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:40,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:40,908 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:40,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,909 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:40,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:40,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:40,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:40,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,910 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:40,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,913 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:40,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:40,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:40,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:40,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:40,913 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:40,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:40,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:41,014 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:41,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:41,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:41,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:41,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,020 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:41,020 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:41,020 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:41,020 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:41,020 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,021 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,022 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,022 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:41,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:41,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:41,022 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,022 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:41,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:41,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,023 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,023 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:41,023 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,023 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:41,023 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:41,023 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:41,024 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:41,024 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:41,024 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:41,024 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:41,024 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:41,437 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228890c0ee9f07f4ac7bc8e19f862d96231 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228890c0ee9f07f4ac7bc8e19f862d96231 2023-03-09 01:54:41,844 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/ca307857aeac4403abc5b5a20df4366b 2023-03-09 01:54:41,849 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/ca307857aeac4403abc5b5a20df4366b as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/ca307857aeac4403abc5b5a20df4366b 2023-03-09 01:54:41,853 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/ca307857aeac4403abc5b5a20df4366b, entries=1, sequenceid=21, filesize=5.0 K 2023-03-09 01:54:41,854 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 830ms, sequenceid=21, compaction requested=true 2023-03-09 01:54:41,854 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:41,854 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:41,856 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,856 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:41,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:41,856 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:41,856 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:41,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:41,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50e787a6[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:41,858 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:41,858 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,863 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,863 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:41,863 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:41,863 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:41,863 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,863 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:41,866 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,866 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:41,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:41,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:41,866 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:41,866 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:41,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:41,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:41,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:41,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:41,868 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:41,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:41,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:41,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,869 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:41,870 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,870 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,870 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:41,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:41,871 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:41,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,872 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:41,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:41,875 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:41,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:41,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:41,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:41,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:41,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,875 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,876 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:41,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:41,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:41,876 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:41,976 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:41,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:41,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:41,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:41,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,984 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:41,984 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:41,984 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:41,984 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:41,984 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,984 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,986 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:41,986 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:41,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:41,986 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:41,986 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:41,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:41,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,987 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:41,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:41,987 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:41,987 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:41,987 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:41,991 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:41,991 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:41,991 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:41,991 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:41,991 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:42,305 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:54:42,407 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231402fa57b1587459d8a221ff447e1db57 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231402fa57b1587459d8a221ff447e1db57 2023-03-09 01:54:42,813 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/70683c18858c4da6a799cc6912f2049a 2023-03-09 01:54:42,818 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/70683c18858c4da6a799cc6912f2049a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/70683c18858c4da6a799cc6912f2049a 2023-03-09 01:54:42,823 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/70683c18858c4da6a799cc6912f2049a, entries=1, sequenceid=25, filesize=5.0 K 2023-03-09 01:54:42,823 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 832ms, sequenceid=25, compaction requested=true 2023-03-09 01:54:42,824 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:42,824 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:42,824 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:42,824 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:42,824 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,824 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:42,824 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:42,826 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,826 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:42,826 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,826 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:42,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:42,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:42,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:42,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11908713[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:42,827 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:42,827 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,829 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,829 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:42,829 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:42,829 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:42,829 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,829 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:42,830 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:42,830 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:42,830 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,830 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:42,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:42,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:42,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:42,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:42,832 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:42,832 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:42,832 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:42,833 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,833 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:42,835 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,835 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:42,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:42,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,835 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:42,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:42,836 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:42,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:42,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:42,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,837 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:42,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:42,839 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:42,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:42,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:42,839 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-09 01:54:42,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:42,839 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,840 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:42,840 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:42,940 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:42,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:42,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:42,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:42,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,947 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:42,947 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:42,947 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:42,947 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:42,947 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,947 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,949 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,949 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:42,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:42,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:42,949 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:42,949 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:42,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,950 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:42,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,950 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,950 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:42,950 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:42,950 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:42,950 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:42,950 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:42,950 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:42,950 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:42,950 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:42,951 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:42,951 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:43,363 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601014b25374ec5844312aaa620fdf03fed47 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601014b25374ec5844312aaa620fdf03fed47 2023-03-09 01:54:43,768 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/e95fdf59014947d69bbc0756c5d8881f 2023-03-09 01:54:43,774 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/e95fdf59014947d69bbc0756c5d8881f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e95fdf59014947d69bbc0756c5d8881f 2023-03-09 01:54:43,779 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e95fdf59014947d69bbc0756c5d8881f, entries=1, sequenceid=29, filesize=5.0 K 2023-03-09 01:54:43,780 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 829ms, sequenceid=29, compaction requested=true 2023-03-09 01:54:43,780 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:43,780 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:43,782 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,782 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:43,784 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:43,784 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fe9f48d[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:43,784 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:43,784 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,786 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,786 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:43,786 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,786 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:43,786 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:43,786 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:43,788 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:43,788 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,788 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:43,788 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:43,791 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:43,791 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:43,791 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:43,791 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:43,791 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,791 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:43,793 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,793 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,793 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:43,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:43,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:43,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:43,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:43,794 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:43,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:43,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,795 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,804 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:43,804 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:43,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:43,805 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:43,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:43,805 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:43,905 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:43,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:43,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:43,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:43,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,912 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:43,912 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:43,912 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:43,912 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:43,912 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,912 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,914 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:43,914 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:43,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:43,914 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:43,914 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:43,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:43,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,915 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:43,915 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:43,916 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:43,916 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:43,916 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:43,916 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:43,916 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:43,916 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:43,916 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:43,916 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:44,329 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010304a56a9d3b444dc38addc33a1662d44c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010304a56a9d3b444dc38addc33a1662d44c 2023-03-09 01:54:44,735 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/bf75f18aab3e49578a7b6ea44221a1ae 2023-03-09 01:54:44,739 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/bf75f18aab3e49578a7b6ea44221a1ae as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/bf75f18aab3e49578a7b6ea44221a1ae 2023-03-09 01:54:44,744 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/bf75f18aab3e49578a7b6ea44221a1ae, entries=1, sequenceid=33, filesize=5.0 K 2023-03-09 01:54:44,745 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 829ms, sequenceid=33, compaction requested=true 2023-03-09 01:54:44,745 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:44,745 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:44,745 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:44,745 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:44,745 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,745 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:44,745 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,747 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:44,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:44,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6841b6b1[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:44,749 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:44,749 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,750 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,751 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:44,751 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,751 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:44,751 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:44,751 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:44,752 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:44,752 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:44,752 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,752 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:44,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:44,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:44,754 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:44,755 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:44,755 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:44,755 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,755 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:44,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:44,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:44,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:44,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,758 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:44,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,760 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:44,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:44,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:44,760 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:44,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:44,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:44,761 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:44,761 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:44,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:44,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:44,861 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:44,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:44,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:44,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:44,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,868 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:44,868 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:44,868 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:44,868 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:44,868 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,868 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,870 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,870 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:44,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:44,871 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:44,871 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:44,872 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:44,872 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:44,872 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:45,284 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907b1ecfe9059574671abd9c86b3813dd01 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907b1ecfe9059574671abd9c86b3813dd01 2023-03-09 01:54:45,691 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/8a55e24a6efb42939977e68b3ad12412 2023-03-09 01:54:45,696 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/8a55e24a6efb42939977e68b3ad12412 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/8a55e24a6efb42939977e68b3ad12412 2023-03-09 01:54:45,700 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/8a55e24a6efb42939977e68b3ad12412, entries=1, sequenceid=37, filesize=5.0 K 2023-03-09 01:54:45,701 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 829ms, sequenceid=37, compaction requested=true 2023-03-09 01:54:45,701 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:45,701 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:45,701 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:45,701 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:45,701 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,701 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:45,701 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:45,703 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,703 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:45,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:45,704 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,704 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:45,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:45,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:45,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:45,705 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:45,705 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@477d6a8[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:45,705 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:45,705 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,706 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,707 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:45,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:45,707 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:45,707 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:45,708 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,708 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:45,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:45,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:45,708 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:45,708 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:45,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:45,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:45,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:45,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:45,711 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:45,711 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:45,711 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:45,711 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:45,711 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,711 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:45,714 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,714 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:45,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,714 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:45,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:45,714 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:45,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:45,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:45,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:45,715 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:45,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,720 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,720 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:45,720 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:45,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:45,720 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:45,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:45,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:45,720 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-09 01:54:45,720 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:45,720 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,721 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,721 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,721 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:45,721 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:45,721 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:45,721 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:45,821 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:45,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:45,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:45,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:45,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,828 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:45,828 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:45,828 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-09 01:54:45,828 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-09 01:54:45,829 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,829 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,830 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:45,830 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:45,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:45,830 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:45,830 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:45,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:45,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,831 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,832 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-09 01:54:45,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-09 01:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-09 01:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:45,832 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:45,832 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. started... 2023-03-09 01:54:45,832 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3a92a0ebc17a6fe227772466246c6162 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:46,246 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112040f5e58414eb491485cf2ae834462efd to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112040f5e58414eb491485cf2ae834462efd 2023-03-09 01:54:46,652 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/a8180cc5cb764792a7b3c6f7f916dd87 2023-03-09 01:54:46,658 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/a8180cc5cb764792a7b3c6f7f916dd87 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/a8180cc5cb764792a7b3c6f7f916dd87 2023-03-09 01:54:46,663 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/a8180cc5cb764792a7b3c6f7f916dd87, entries=1, sequenceid=41, filesize=5.0 K 2023-03-09 01:54:46,664 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3a92a0ebc17a6fe227772466246c6162 in 832ms, sequenceid=41, compaction requested=true 2023-03-09 01:54:46,664 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:46,664 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:46,664 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:46,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:46,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-09 01:54:46,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-09 01:54:46,668 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,668 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:46,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:46,668 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,668 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:46,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:46,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:46,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:46,669 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-09 01:54:46,669 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11ca712a[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:46,669 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-09 01:54:46,669 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,670 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,671 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-09 01:54:46,671 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-09 01:54:46,671 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,671 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:46,671 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:46,672 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,672 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-09 01:54:46,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:46,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:46,672 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:46,672 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-09 01:54:46,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:46,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:46,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-09 01:54:46,674 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-09 01:54:46,674 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:46,674 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:46,674 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,675 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:46,676 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,676 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:46,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:46,676 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:46,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:46,677 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,678 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:46,681 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:46,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-09 01:54:46,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:46,681 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:46,682 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:46,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:46,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:46,782 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:46,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:46,784 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:46,786 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:46,786 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:46,787 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:46,787 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327486787 2023-03-09 01:54:46,788 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:54:46,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-09 01:54:46,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:46,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x49d884fb to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:54:46,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@378fa920, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:54:46,794 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,631 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103fcd207b48d3a4945ac555e1ebd68ac9d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103fcd207b48d3a4945ac555e1ebd68ac9d 2023-03-09 01:54:47,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:47,645 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:54:47,646 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34184, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:54:47,653 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:54:47,655 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46250, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:54:47,663 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bc94e1df035f41728c681daab9f4b674 first=Optional[r3] last=Optional[r6] 2023-03-09 01:54:47,664 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bc94e1df035f41728c681daab9f4b674}] 2023-03-09 01:54:47,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bc94e1df035f41728c681daab9f4b674 for inclusion in 3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:47,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-09 01:54:47,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:47,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:47,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bc94e1df035f41728c681daab9f4b674 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2vdhmle2b9f02inasbl9k5intp2bej2geie5ph1jr6kc0scsslduh7s7kh003j7e/family1/bc94e1df035f41728c681daab9f4b674 2023-03-09 01:54:47,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2vdhmle2b9f02inasbl9k5intp2bej2geie5ph1jr6kc0scsslduh7s7kh003j7e/family1/bc94e1df035f41728c681daab9f4b674 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e9a6aec67c74409a9f06354ed0478a85_SeqId_44_ 2023-03-09 01:54:47,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2vdhmle2b9f02inasbl9k5intp2bej2geie5ph1jr6kc0scsslduh7s7kh003j7e/family1/bc94e1df035f41728c681daab9f4b674 into 3a92a0ebc17a6fe227772466246c6162/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e9a6aec67c74409a9f06354ed0478a85_SeqId_44_ - updating store file list. 2023-03-09 01:54:47,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e9a6aec67c74409a9f06354ed0478a85_SeqId_44_ into 3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:47,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2vdhmle2b9f02inasbl9k5intp2bej2geie5ph1jr6kc0scsslduh7s7kh003j7e/family1/bc94e1df035f41728c681daab9f4b674 into 3a92a0ebc17a6fe227772466246c6162/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e9a6aec67c74409a9f06354ed0478a85_SeqId_44_) 2023-03-09 01:54:47,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2vdhmle2b9f02inasbl9k5intp2bej2geie5ph1jr6kc0scsslduh7s7kh003j7e/family1/bc94e1df035f41728c681daab9f4b674 2023-03-09 01:54:47,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-09 01:54:47,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-09 01:54:47,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 3a92a0ebc17a6fe227772466246c6162/family1 is initiating minor compaction 2023-03-09 01:54:47,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7c8aed9a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:47,681 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3a92a0ebc17a6fe227772466246c6162/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:47,682 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4837b65730e24c8e8fc8eafd65a8a0ce, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4c2ae58284264883839d0de8aa5e6d01, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/873e5dcff97244438f99ef299831ac00, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/ca307857aeac4403abc5b5a20df4366b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/70683c18858c4da6a799cc6912f2049a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e95fdf59014947d69bbc0756c5d8881f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/bf75f18aab3e49578a7b6ea44221a1ae, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/8a55e24a6efb42939977e68b3ad12412, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/a8180cc5cb764792a7b3c6f7f916dd87, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e9a6aec67c74409a9f06354ed0478a85_SeqId_44_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp, totalSize=50.4 K 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 4837b65730e24c8e8fc8eafd65a8a0ce, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 4c2ae58284264883839d0de8aa5e6d01, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 873e5dcff97244438f99ef299831ac00, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting ca307857aeac4403abc5b5a20df4366b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 70683c18858c4da6a799cc6912f2049a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting e95fdf59014947d69bbc0756c5d8881f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting bf75f18aab3e49578a7b6ea44221a1ae, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 8a55e24a6efb42939977e68b3ad12412, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting a8180cc5cb764792a7b3c6f7f916dd87, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-09 01:54:47,682 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting e9a6aec67c74409a9f06354ed0478a85_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-09 01:54:47,684 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:47,684 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:47,686 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228890c0ee9f07f4ac7bc8e19f862d96231 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228890c0ee9f07f4ac7bc8e19f862d96231 2023-03-09 01:54:47,687 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231402fa57b1587459d8a221ff447e1db57 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231402fa57b1587459d8a221ff447e1db57 2023-03-09 01:54:47,689 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601014b25374ec5844312aaa620fdf03fed47 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601014b25374ec5844312aaa620fdf03fed47 2023-03-09 01:54:47,690 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010304a56a9d3b444dc38addc33a1662d44c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010304a56a9d3b444dc38addc33a1662d44c 2023-03-09 01:54:47,690 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-09 01:54:47,690 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-09 01:54:47,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:47,714 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3a92a0ebc17a6fe227772466246c6162#family1#compaction#60 average throughput is 0.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:54:48,538 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/cdc0a847ed3e4cb3975428fed1191d3e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/cdc0a847ed3e4cb3975428fed1191d3e 2023-03-09 01:54:48,544 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 3a92a0ebc17a6fe227772466246c6162/family1 of 3a92a0ebc17a6fe227772466246c6162 into cdc0a847ed3e4cb3975428fed1191d3e(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:54:48,544 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:48,544 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678326887681; duration=0sec 2023-03-09 01:54:48,545 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:48,563 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054397465d33ee415ba7f91b46cdd2c3e3 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512054397465d33ee415ba7f91b46cdd2c3e3 2023-03-09 01:54:48,564 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:48,588 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0cfc6c7e25734b65b43de2e015654fe6 first=Optional[r00] last=Optional[r2] 2023-03-09 01:54:48,588 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0cfc6c7e25734b65b43de2e015654fe6}] 2023-03-09 01:54:48,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0cfc6c7e25734b65b43de2e015654fe6 for inclusion in 3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:48,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-09 01:54:48,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:48,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:48,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0cfc6c7e25734b65b43de2e015654fe6 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__isrmvr7hfmg1iqj33726u31ke0o655govbg42altrnh9vqim6tuivgn4ig4nh54g/family1/0cfc6c7e25734b65b43de2e015654fe6 2023-03-09 01:54:48,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__isrmvr7hfmg1iqj33726u31ke0o655govbg42altrnh9vqim6tuivgn4ig4nh54g/family1/0cfc6c7e25734b65b43de2e015654fe6 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/af10b3d64dc14138b533536dae1a74c4_SeqId_47_ 2023-03-09 01:54:48,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__isrmvr7hfmg1iqj33726u31ke0o655govbg42altrnh9vqim6tuivgn4ig4nh54g/family1/0cfc6c7e25734b65b43de2e015654fe6 into 3a92a0ebc17a6fe227772466246c6162/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/af10b3d64dc14138b533536dae1a74c4_SeqId_47_ - updating store file list. 2023-03-09 01:54:48,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/af10b3d64dc14138b533536dae1a74c4_SeqId_47_ into 3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:48,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__isrmvr7hfmg1iqj33726u31ke0o655govbg42altrnh9vqim6tuivgn4ig4nh54g/family1/0cfc6c7e25734b65b43de2e015654fe6 into 3a92a0ebc17a6fe227772466246c6162/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/af10b3d64dc14138b533536dae1a74c4_SeqId_47_) 2023-03-09 01:54:48,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__isrmvr7hfmg1iqj33726u31ke0o655govbg42altrnh9vqim6tuivgn4ig4nh54g/family1/0cfc6c7e25734b65b43de2e015654fe6 2023-03-09 01:54:48,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-09 01:54:48,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-09 01:54:48,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 3a92a0ebc17a6fe227772466246c6162/family1 is initiating minor compaction (all files) 2023-03-09 01:54:48,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@54ebf87a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:48,603 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3a92a0ebc17a6fe227772466246c6162/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:48,603 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/55e78b03425a488086bd6b1f021a582a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/cdc0a847ed3e4cb3975428fed1191d3e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/af10b3d64dc14138b533536dae1a74c4_SeqId_47_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp, totalSize=390.1 K 2023-03-09 01:54:48,604 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 55e78b03425a488086bd6b1f021a582a, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843359 2023-03-09 01:54:48,604 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting cdc0a847ed3e4cb3975428fed1191d3e, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243359 2023-03-09 01:54:48,605 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting af10b3d64dc14138b533536dae1a74c4_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878843359 2023-03-09 01:54:48,605 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:48,606 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:48,619 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb7be2ff6a1a46e7a42a2e50d692cd95 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb7be2ff6a1a46e7a42a2e50d692cd95 2023-03-09 01:54:48,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4837b65730e24c8e8fc8eafd65a8a0ce, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4c2ae58284264883839d0de8aa5e6d01, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/873e5dcff97244438f99ef299831ac00, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/ca307857aeac4403abc5b5a20df4366b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/70683c18858c4da6a799cc6912f2049a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e95fdf59014947d69bbc0756c5d8881f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/bf75f18aab3e49578a7b6ea44221a1ae, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/8a55e24a6efb42939977e68b3ad12412, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/a8180cc5cb764792a7b3c6f7f916dd87, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e9a6aec67c74409a9f06354ed0478a85_SeqId_44_] to archive 2023-03-09 01:54:48,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:48,621 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201e331fd7f3a9b4921af959b3deaf081ba to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201e331fd7f3a9b4921af959b3deaf081ba 2023-03-09 01:54:48,622 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205e691d91b1f76408f89ade10976166129 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205e691d91b1f76408f89ade10976166129 2023-03-09 01:54:48,622 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-09 01:54:48,622 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-09 01:54:48,623 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4837b65730e24c8e8fc8eafd65a8a0ce to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4837b65730e24c8e8fc8eafd65a8a0ce 2023-03-09 01:54:48,623 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:48,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:48,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:48,624 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:48,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4c2ae58284264883839d0de8aa5e6d01 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/4c2ae58284264883839d0de8aa5e6d01 2023-03-09 01:54:48,626 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8390 sec 2023-03-09 01:54:48,627 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/873e5dcff97244438f99ef299831ac00 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/873e5dcff97244438f99ef299831ac00 2023-03-09 01:54:48,629 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/ca307857aeac4403abc5b5a20df4366b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/ca307857aeac4403abc5b5a20df4366b 2023-03-09 01:54:48,630 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/70683c18858c4da6a799cc6912f2049a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/70683c18858c4da6a799cc6912f2049a 2023-03-09 01:54:48,632 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e95fdf59014947d69bbc0756c5d8881f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e95fdf59014947d69bbc0756c5d8881f 2023-03-09 01:54:48,634 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/bf75f18aab3e49578a7b6ea44221a1ae to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/bf75f18aab3e49578a7b6ea44221a1ae 2023-03-09 01:54:48,635 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/8a55e24a6efb42939977e68b3ad12412 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/8a55e24a6efb42939977e68b3ad12412 2023-03-09 01:54:48,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/a8180cc5cb764792a7b3c6f7f916dd87 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/a8180cc5cb764792a7b3c6f7f916dd87 2023-03-09 01:54:48,662 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:54:48,663 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3a92a0ebc17a6fe227772466246c6162#family1#compaction#61 average throughput is 5.02 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:54:48,663 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e9a6aec67c74409a9f06354ed0478a85_SeqId_44_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e9a6aec67c74409a9f06354ed0478a85_SeqId_44_ 2023-03-09 01:54:48,663 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x49d884fb to 127.0.0.1:50548 2023-03-09 01:54:48,663 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:54:48,664 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:54:48,664 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x201ff8b0 to 127.0.0.1:50548 2023-03-09 01:54:48,664 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:54:48,664 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:54:48,664 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3dcdead5 to 127.0.0.1:50548 2023-03-09 01:54:48,664 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:54:48,664 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:54:48,664 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x32a91364 to 127.0.0.1:50548 2023-03-09 01:54:48,664 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:54:48,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-09 01:54:48,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:49,139 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-09 01:54:49,140 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-09 01:54:49,140 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-09 01:54:49,142 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-09 01:54:49,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, REOPEN/MOVE}] 2023-03-09 01:54:49,146 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, REOPEN/MOVE 2023-03-09 01:54:49,146 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:49,146 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326889146"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326889146"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326889146"}]},"ts":"1678326889146"} 2023-03-09 01:54:49,148 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:49,299 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3a92a0ebc17a6fe227772466246c6162, disabling compactions & flushes 2023-03-09 01:54:49,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:49,486 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/.tmp/family1/e24b57a32d6a4c2aba18c928f47f9408 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e24b57a32d6a4c2aba18c928f47f9408 2023-03-09 01:54:49,490 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3a92a0ebc17a6fe227772466246c6162/family1 of 3a92a0ebc17a6fe227772466246c6162 into e24b57a32d6a4c2aba18c928f47f9408(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:54:49,490 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:49,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:49,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:49,490 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1678326888603; duration=0sec 2023-03-09 01:54:49,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. after waiting 0 ms 2023-03-09 01:54:49,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:49,491 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:54:49,491 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/55e78b03425a488086bd6b1f021a582a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/cdc0a847ed3e4cb3975428fed1191d3e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/af10b3d64dc14138b533536dae1a74c4_SeqId_47_] to archive 2023-03-09 01:54:49,492 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:49,493 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/55e78b03425a488086bd6b1f021a582a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/55e78b03425a488086bd6b1f021a582a 2023-03-09 01:54:49,494 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/cdc0a847ed3e4cb3975428fed1191d3e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/cdc0a847ed3e4cb3975428fed1191d3e 2023-03-09 01:54:49,495 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/af10b3d64dc14138b533536dae1a74c4_SeqId_47_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/af10b3d64dc14138b533536dae1a74c4_SeqId_47_ 2023-03-09 01:54:49,500 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-09 01:54:49,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:49,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:49,501 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 3a92a0ebc17a6fe227772466246c6162 to self. 2023-03-09 01:54:49,503 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,503 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=CLOSED 2023-03-09 01:54:49,503 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326889503"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326889503"}]},"ts":"1678326889503"} 2023-03-09 01:54:49,505 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-09 01:54:49,505 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757 in 356 msec 2023-03-09 01:54:49,506 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=true 2023-03-09 01:54:49,656 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:49,657 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326889656"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326889656"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326889656"}]},"ts":"1678326889656"} 2023-03-09 01:54:49,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:49,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:49,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3a92a0ebc17a6fe227772466246c6162, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:54:49,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:49,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,815 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,816 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:49,816 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:49,816 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3a92a0ebc17a6fe227772466246c6162 columnFamilyName family1 2023-03-09 01:54:49,821 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e24b57a32d6a4c2aba18c928f47f9408 2023-03-09 01:54:49,821 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(310): Store=3a92a0ebc17a6fe227772466246c6162/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:49,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:49,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3a92a0ebc17a6fe227772466246c6162; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60327896, jitterRate=-0.10104429721832275}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:54:49,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:49,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162., pid=47, masterSystemTime=1678326889810 2023-03-09 01:54:49,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:49,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:49,829 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:49,829 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326889829"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326889829"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326889829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326889829"}]},"ts":"1678326889829"} 2023-03-09 01:54:49,832 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-09 01:54:49,832 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757 in 173 msec 2023-03-09 01:54:49,833 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-09 01:54:49,833 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, REOPEN/MOVE in 687 msec 2023-03-09 01:54:49,835 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-09 01:54:49,835 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 692 msec 2023-03-09 01:54:49,837 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1070 sec 2023-03-09 01:54:49,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-09 01:54:49,838 INFO [Listener at localhost/33973] compactions.TestMobCompactor(278): alter status finished 2023-03-09 01:54:49,839 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:49,840 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:49,841 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:49,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:49,842 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327489842 2023-03-09 01:54:49,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:54:49,843 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-09 01:54:49,843 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:49,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x0e27fd0a to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:54:49,848 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@442b8347, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:54:49,849 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:49,849 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:49,849 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:49,849 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:49,849 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:50,680 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f16ee622a98048e99bf0bc20f2ed99c0 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128f16ee622a98048e99bf0bc20f2ed99c0 2023-03-09 01:54:50,680 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:50,693 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:54:50,694 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34196, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:54:50,700 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:54:50,702 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46256, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:54:50,710 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ecbb67394bd9423d92d530c3774977e9 first=Optional[r05] last=Optional[r8] 2023-03-09 01:54:50,711 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ecbb67394bd9423d92d530c3774977e9}] 2023-03-09 01:54:50,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ecbb67394bd9423d92d530c3774977e9 for inclusion in 3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:50,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-09 01:54:50,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:54:50,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:50,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ecbb67394bd9423d92d530c3774977e9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__20av6glca6mdt81qdmrdcr5ipvprrge3agh3r1de8gg8ih9a4bedscnmri7k1hpc/family1/ecbb67394bd9423d92d530c3774977e9 2023-03-09 01:54:50,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__20av6glca6mdt81qdmrdcr5ipvprrge3agh3r1de8gg8ih9a4bedscnmri7k1hpc/family1/ecbb67394bd9423d92d530c3774977e9 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/42d97e51294c4da08f5fd43acfc6cec8_SeqId_53_ 2023-03-09 01:54:50,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__20av6glca6mdt81qdmrdcr5ipvprrge3agh3r1de8gg8ih9a4bedscnmri7k1hpc/family1/ecbb67394bd9423d92d530c3774977e9 into 3a92a0ebc17a6fe227772466246c6162/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/42d97e51294c4da08f5fd43acfc6cec8_SeqId_53_ - updating store file list. 2023-03-09 01:54:50,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/42d97e51294c4da08f5fd43acfc6cec8_SeqId_53_ into 3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:50,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__20av6glca6mdt81qdmrdcr5ipvprrge3agh3r1de8gg8ih9a4bedscnmri7k1hpc/family1/ecbb67394bd9423d92d530c3774977e9 into 3a92a0ebc17a6fe227772466246c6162/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/42d97e51294c4da08f5fd43acfc6cec8_SeqId_53_) 2023-03-09 01:54:50,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__20av6glca6mdt81qdmrdcr5ipvprrge3agh3r1de8gg8ih9a4bedscnmri7k1hpc/family1/ecbb67394bd9423d92d530c3774977e9 2023-03-09 01:54:50,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:50,737 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:54:50,738 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112040f5e58414eb491485cf2ae834462efd to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112040f5e58414eb491485cf2ae834462efd 2023-03-09 01:54:50,740 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128399d00cb74b344859fc713c01f61d25f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128399d00cb74b344859fc713c01f61d25f 2023-03-09 01:54:50,740 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-09 01:54:50,740 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-09 01:54:50,740 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:50,740 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:50,740 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:50,740 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:50,740 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:50,742 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:50,743 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 902 msec 2023-03-09 01:54:50,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-09 01:54:50,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:51,210 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-09 01:54:51,211 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-09 01:54:51,211 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-09 01:54:51,214 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-09 01:54:51,216 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, REOPEN/MOVE}] 2023-03-09 01:54:51,217 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, REOPEN/MOVE 2023-03-09 01:54:51,218 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:51,218 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326891218"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326891218"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326891218"}]},"ts":"1678326891218"} 2023-03-09 01:54:51,219 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:51,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3a92a0ebc17a6fe227772466246c6162, disabling compactions & flushes 2023-03-09 01:54:51,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:51,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:51,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. after waiting 0 ms 2023-03-09 01:54:51,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:51,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-09 01:54:51,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:51,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:51,378 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 3a92a0ebc17a6fe227772466246c6162 to self. 2023-03-09 01:54:51,380 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,380 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=CLOSED 2023-03-09 01:54:51,380 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326891380"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326891380"}]},"ts":"1678326891380"} 2023-03-09 01:54:51,382 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-09 01:54:51,383 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757 in 162 msec 2023-03-09 01:54:51,383 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=true 2023-03-09 01:54:51,533 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:51,534 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326891533"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326891533"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326891533"}]},"ts":"1678326891533"} 2023-03-09 01:54:51,535 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:51,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:51,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3a92a0ebc17a6fe227772466246c6162, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:54:51,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:51,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,691 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,692 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:51,692 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1 2023-03-09 01:54:51,693 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3a92a0ebc17a6fe227772466246c6162 columnFamilyName family1 2023-03-09 01:54:51,698 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/42d97e51294c4da08f5fd43acfc6cec8_SeqId_53_ 2023-03-09 01:54:51,701 DEBUG [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/family1/e24b57a32d6a4c2aba18c928f47f9408 2023-03-09 01:54:51,701 INFO [StoreOpener-3a92a0ebc17a6fe227772466246c6162-1] regionserver.HStore(310): Store=3a92a0ebc17a6fe227772466246c6162/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:51,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:54:51,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3a92a0ebc17a6fe227772466246c6162; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60485943, jitterRate=-0.09868921339511871}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:54:51,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:54:51,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162., pid=53, masterSystemTime=1678326891687 2023-03-09 01:54:51,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:51,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:54:51,709 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=3a92a0ebc17a6fe227772466246c6162, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:51,709 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1678326891709"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326891709"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326891709"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326891709"}]},"ts":"1678326891709"} 2023-03-09 01:54:51,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-09 01:54:51,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 3a92a0ebc17a6fe227772466246c6162, server=jenkins-hbase4.apache.org,46511,1678326819757 in 176 msec 2023-03-09 01:54:51,714 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-09 01:54:51,714 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=3a92a0ebc17a6fe227772466246c6162, REOPEN/MOVE in 496 msec 2023-03-09 01:54:51,716 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-09 01:54:51,716 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 500 msec 2023-03-09 01:54:51,718 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 916 msec 2023-03-09 01:54:51,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-09 01:54:51,720 INFO [Listener at localhost/33973] compactions.TestMobCompactor(278): alter status finished 2023-03-09 01:54:51,721 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-09 01:54:51,722 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:51,722 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:51,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:51,723 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327491723 2023-03-09 01:54:51,724 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:54:51,724 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-09 01:54:51,724 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:51,725 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x220bc4f3 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:54:51,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a225a4e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:54:51,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:51,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:51,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:51,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:51,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:51,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:51,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:51,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-09 01:54:51,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-09 01:54:51,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:54:51,732 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-09 01:54:51,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 11 msec 2023-03-09 01:54:51,781 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=237 (was 264), OpenFileDescriptor=596 (was 628), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 202), ProcessCount=171 (was 171), AvailableMemoryMB=8093 (was 8137) 2023-03-09 01:54:51,792 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=237, OpenFileDescriptor=596, MaxFileDescriptor=60000, SystemLoadAverage=173, ProcessCount=171, AvailableMemoryMB=8092 2023-03-09 01:54:51,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-09 01:54:51,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:51,796 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:54:51,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-09 01:54:51,797 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:54:51,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-09 01:54:51,798 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:51,799 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 empty. 2023-03-09 01:54:51,799 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:51,799 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-09 01:54:51,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-09 01:54:52,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-09 01:54:52,207 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-09 01:54:52,209 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 527ef65d68bd3fd8127836853154b8b0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:54:52,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-09 01:54:52,615 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:52,615 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 527ef65d68bd3fd8127836853154b8b0, disabling compactions & flushes 2023-03-09 01:54:52,615 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:52,615 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:52,615 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. after waiting 0 ms 2023-03-09 01:54:52,615 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:52,615 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:52,615 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:54:52,619 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:54:52,620 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326892619"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326892619"}]},"ts":"1678326892619"} 2023-03-09 01:54:52,621 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:54:52,622 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:54:52,622 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326892622"}]},"ts":"1678326892622"} 2023-03-09 01:54:52,623 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-09 01:54:52,626 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, ASSIGN}] 2023-03-09 01:54:52,627 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, ASSIGN 2023-03-09 01:54:52,628 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:54:52,779 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:52,779 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326892779"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326892779"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326892779"}]},"ts":"1678326892779"} 2023-03-09 01:54:52,781 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:54:52,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-09 01:54:52,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:52,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 527ef65d68bd3fd8127836853154b8b0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:54:52,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:52,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:54:52,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:52,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:52,938 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:52,939 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:54:52,939 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:54:52,940 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 527ef65d68bd3fd8127836853154b8b0 columnFamilyName family1 2023-03-09 01:54:52,941 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(310): Store=527ef65d68bd3fd8127836853154b8b0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:54:52,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:52,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:52,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:54:52,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:54:52,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 527ef65d68bd3fd8127836853154b8b0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64503519, jitterRate=-0.03882266581058502}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:54:52,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:54:52,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0., pid=57, masterSystemTime=1678326892932 2023-03-09 01:54:52,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:52,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:52,950 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:52,950 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326892950"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326892950"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326892950"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326892950"}]},"ts":"1678326892950"} 2023-03-09 01:54:52,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-09 01:54:52,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757 in 170 msec 2023-03-09 01:54:52,954 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-09 01:54:52,954 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, ASSIGN in 326 msec 2023-03-09 01:54:52,954 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:54:52,954 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326892954"}]},"ts":"1678326892954"} 2023-03-09 01:54:52,955 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-09 01:54:52,958 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:54:52,959 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1640 sec 2023-03-09 01:54:53,556 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:53,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-09 01:54:53,903 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-09 01:54:53,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:53,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:53,930 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:53,930 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:53,930 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:54:53,930 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:54:53,930 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:53,930 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:53,938 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:53,938 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:53,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:53,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:53,939 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:53,939 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:53,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:53,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:53,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:53,940 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:53,940 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:54:53,941 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-09 01:54:54,365 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113029606e22716248c49f52bed13d44452f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113029606e22716248c49f52bed13d44452f 2023-03-09 01:54:54,770 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/e99ae4d851cf48c5b0dc5e7889b7a7a1 2023-03-09 01:54:54,774 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/e99ae4d851cf48c5b0dc5e7889b7a7a1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e99ae4d851cf48c5b0dc5e7889b7a7a1 2023-03-09 01:54:54,778 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e99ae4d851cf48c5b0dc5e7889b7a7a1, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-09 01:54:54,779 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 838ms, sequenceid=5, compaction requested=false 2023-03-09 01:54:54,780 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:54:54,780 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:54,780 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:54,780 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:54,780 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,780 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:54:54,780 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:54:54,782 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,782 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:54,782 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,782 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:54,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:54,783 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:54:54,783 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41d7b202[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:54,783 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:54:54,784 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,785 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,785 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:54:54,785 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,785 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:54,785 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:54:54,785 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:54,787 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:54:54,787 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,787 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:54,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:54,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:54,787 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:54:54,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:54,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:54,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:54,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:54,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:54:54,789 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:54:54,789 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:54,789 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:54,789 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,789 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:54,796 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,796 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:54,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,796 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:54,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:54,796 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,797 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,801 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,801 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:54,801 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:54,801 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:54,802 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:54,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-09 01:54:54,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:54,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:54,802 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,802 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:54,802 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,802 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,802 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,802 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:54,802 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:54,902 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:54,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:54,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:54,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:54,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,909 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:54,909 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:54,909 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:54:54,909 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:54:54,909 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,909 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,911 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,911 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:54,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:54,911 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:54,911 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:54,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,912 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:54:54,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:54,912 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:54:54,912 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:54,912 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:54:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:54,913 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:54,913 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:54:54,913 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-09 01:54:55,032 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:54:55,325 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511288be8f660d096418b9b321aba56b61d7c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288be8f660d096418b9b321aba56b61d7c 2023-03-09 01:54:55,730 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/bd7888532fb2471389b418ca6fca85d8 2023-03-09 01:54:55,735 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/bd7888532fb2471389b418ca6fca85d8 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/bd7888532fb2471389b418ca6fca85d8 2023-03-09 01:54:55,739 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/bd7888532fb2471389b418ca6fca85d8, entries=1, sequenceid=9, filesize=5.0 K 2023-03-09 01:54:55,740 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 827ms, sequenceid=9, compaction requested=false 2023-03-09 01:54:55,740 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:54:55,740 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:55,740 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:55,740 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:55,740 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,740 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:54:55,740 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:54:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,743 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:55,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:55,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:54:55,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@244660b7[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:55,744 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:54:55,744 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,745 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:54:55,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:54:55,746 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:55,746 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:55,747 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,747 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:54:55,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:55,747 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:55,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:55,747 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:54:55,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:55,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:55,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:55,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:55,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:54:55,749 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:54:55,749 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:55,749 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:55,749 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,749 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:55,751 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,751 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:55,751 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:55,751 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:55,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:55,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:55,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,753 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:55,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:55,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:55,756 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-09 01:54:55,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:55,756 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:55,757 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:55,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:55,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:55,857 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:55,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:55,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:55,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:55,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,863 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:55,863 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:55,863 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:54:55,863 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:54:55,864 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,864 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,866 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:55,866 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:55,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:55,866 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:55,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,866 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:55,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:54:55,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:55,867 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:54:55,867 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:55,867 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:54:55,868 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:55,868 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:55,868 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:55,868 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:54:55,868 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:56,281 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120153cd58d125ca48e286133ef7f2091062 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120153cd58d125ca48e286133ef7f2091062 2023-03-09 01:54:56,687 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/afd59a9090274183ba7706ba31728eaf 2023-03-09 01:54:56,692 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/afd59a9090274183ba7706ba31728eaf as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/afd59a9090274183ba7706ba31728eaf 2023-03-09 01:54:56,696 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/afd59a9090274183ba7706ba31728eaf, entries=1, sequenceid=13, filesize=5.0 K 2023-03-09 01:54:56,697 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 829ms, sequenceid=13, compaction requested=true 2023-03-09 01:54:56,697 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:54:56,697 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:56,697 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:56,697 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:56,697 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,697 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:54:56,697 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:54:56,699 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:56,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,700 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:56,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:56,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:56,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:56,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:54:56,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ebe2001[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:56,701 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:54:56,701 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,702 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,702 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:54:56,702 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:54:56,702 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:56,702 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,702 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:56,704 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,704 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:54:56,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:56,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:56,704 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:56,705 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:54:56,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:56,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:56,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:56,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:56,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:54:56,707 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:54:56,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:56,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:56,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,707 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:56,708 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,708 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,708 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:56,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:56,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:56,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:56,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:56,709 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:56,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:56,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,710 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:56,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,712 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:56,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:56,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:56,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:56,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:56,712 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:56,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:56,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:56,812 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:56,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:56,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:56,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:56,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,819 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:56,819 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:56,819 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:54:56,819 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:54:56,820 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,820 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,821 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:56,821 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:56,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:56,822 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:56,822 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:56,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,822 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:56,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,822 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:54:56,823 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:56,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:54:56,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:56,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:54:56,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:56,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:56,823 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:56,823 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:54:56,823 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:57,235 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c82e3f8bd0de4020b137e73d254c7604 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205c82e3f8bd0de4020b137e73d254c7604 2023-03-09 01:54:57,641 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/e7afa8377786468f9e09ed221cd2e155 2023-03-09 01:54:57,645 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/e7afa8377786468f9e09ed221cd2e155 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e7afa8377786468f9e09ed221cd2e155 2023-03-09 01:54:57,649 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e7afa8377786468f9e09ed221cd2e155, entries=1, sequenceid=17, filesize=5.0 K 2023-03-09 01:54:57,650 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 827ms, sequenceid=17, compaction requested=true 2023-03-09 01:54:57,650 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:54:57,650 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:57,650 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:57,650 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:57,650 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,650 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:54:57,650 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:54:57,659 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,659 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:57,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:57,659 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,660 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:57,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:57,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:57,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:57,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:54:57,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c3f51dd[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:57,661 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:54:57,661 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,671 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,671 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:54:57,671 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:54:57,671 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,671 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:57,671 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:57,678 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:54:57,678 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,678 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:57,679 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:54:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:57,681 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:54:57,681 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:54:57,681 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:57,681 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:57,681 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,681 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:57,688 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,688 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,688 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:57,689 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:57,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,690 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,697 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,697 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:57,697 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,697 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:57,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:57,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:57,697 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:57,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-09 01:54:57,697 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:57,697 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,698 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:57,698 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:57,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:57,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:57,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,798 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:57,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:57,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:57,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:57,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,805 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:57,805 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:57,805 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:54:57,805 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:54:57,806 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,806 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,807 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,807 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:57,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:57,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:57,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:57,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:57,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:54:57,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:57,809 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:57,809 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:54:57,809 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:58,222 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d310e94d9b464fdbbaad3f4783fe76aa to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d310e94d9b464fdbbaad3f4783fe76aa 2023-03-09 01:54:58,628 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/852aae59f1804b6188ae8a67410f4374 2023-03-09 01:54:58,634 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/852aae59f1804b6188ae8a67410f4374 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/852aae59f1804b6188ae8a67410f4374 2023-03-09 01:54:58,638 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/852aae59f1804b6188ae8a67410f4374, entries=1, sequenceid=21, filesize=5.0 K 2023-03-09 01:54:58,638 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 829ms, sequenceid=21, compaction requested=true 2023-03-09 01:54:58,639 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:54:58,639 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:58,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:58,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:58,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:54:58,639 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:54:58,640 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:58,640 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:58,641 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:58,641 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:58,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:58,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:58,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:54:58,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2004bffa[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:58,642 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:54:58,642 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,645 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,645 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,645 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:58,645 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:54:58,645 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:54:58,645 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:58,646 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,646 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:54:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:58,646 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:58,647 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:54:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:58,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:54:58,649 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:54:58,649 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:58,649 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:58,649 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,649 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:58,651 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,651 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:58,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:58,651 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:58,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,651 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:58,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:58,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,652 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,654 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,654 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,654 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:58,654 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:58,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:58,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:58,654 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:58,654 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,655 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,655 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,655 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,654 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:58,655 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:58,655 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:58,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-09 01:54:58,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:58,755 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:58,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:58,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:58,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:58,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,762 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:58,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:58,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:54:58,762 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:54:58,763 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,763 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,764 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,765 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:58,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:58,765 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:58,765 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:58,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:58,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:54:58,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:58,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:54:58,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:58,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:54:58,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:58,766 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:58,766 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:58,766 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:54:58,766 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:54:59,179 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512319371952b8c3642be996f0c7f4ac0e263 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512319371952b8c3642be996f0c7f4ac0e263 2023-03-09 01:54:59,584 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/fc6e0c31b4b5475aa430a76fd9768735 2023-03-09 01:54:59,589 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/fc6e0c31b4b5475aa430a76fd9768735 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fc6e0c31b4b5475aa430a76fd9768735 2023-03-09 01:54:59,592 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fc6e0c31b4b5475aa430a76fd9768735, entries=1, sequenceid=25, filesize=5.0 K 2023-03-09 01:54:59,593 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 827ms, sequenceid=25, compaction requested=true 2023-03-09 01:54:59,593 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:54:59,593 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:59,593 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:54:59,593 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:54:59,593 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,593 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:54:59,593 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:54:59,595 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,595 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:59,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:59,595 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,595 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:54:59,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:59,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:59,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:59,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:54:59,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37d02095[Count = 0] remaining members to acquire global barrier 2023-03-09 01:54:59,596 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:54:59,596 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,597 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,597 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:54:59,597 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:54:59,598 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:54:59,598 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,598 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:54:59,599 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:54:59,599 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,599 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:59,599 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:59,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:59,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:59,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:54:59,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:54:59,601 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:54:59,601 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:54:59,601 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:54:59,601 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,601 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:54:59,603 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,603 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:54:59,603 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:59,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:54:59,604 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:54:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,610 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:54:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,615 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:54:59,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:59,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:59,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,615 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,616 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:54:59,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:54:59,616 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:54:59,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:54:59,716 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:54:59,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:54:59,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:54:59,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:54:59,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,722 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:54:59,722 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:59,722 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:54:59,722 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:54:59,722 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,722 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,727 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,727 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:54:59,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:54:59,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:54:59,727 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:54:59,727 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:54:59,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,727 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:54:59,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,728 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,728 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:54:59,728 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:54:59,728 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:54:59,728 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:54:59,728 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:54:59,728 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:54:59,728 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:54:59,728 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:54:59,729 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:54:59,729 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:55:00,143 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010159dbc52de55e4259896213d5ad166384 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010159dbc52de55e4259896213d5ad166384 2023-03-09 01:55:00,550 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/b70eccaa49e94878a98e212093d7ea72 2023-03-09 01:55:00,555 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/b70eccaa49e94878a98e212093d7ea72 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/b70eccaa49e94878a98e212093d7ea72 2023-03-09 01:55:00,560 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/b70eccaa49e94878a98e212093d7ea72, entries=1, sequenceid=29, filesize=5.0 K 2023-03-09 01:55:00,561 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 832ms, sequenceid=29, compaction requested=true 2023-03-09 01:55:00,561 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:00,561 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:00,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:55:00,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:55:00,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:55:00,561 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:55:00,564 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,564 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:00,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:00,564 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,565 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:00,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:00,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:00,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:00,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:55:00,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5801a046[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:00,566 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:55:00,566 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,567 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,567 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:55:00,567 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:55:00,567 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,567 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:00,567 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:00,569 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:00,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:00,569 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:55:00,569 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:00,569 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:55:00,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:00,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:00,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:00,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:00,571 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:55:00,571 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:55:00,571 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:00,571 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:00,571 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,571 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:00,573 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,573 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:00,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:00,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,573 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:00,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:00,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:00,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:00,573 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:00,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:00,579 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:00,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:00,582 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:00,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:55:00,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:00,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:00,582 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:00,683 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:00,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:00,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:00,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:00,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,690 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:55:00,690 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:00,690 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:55:00,690 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:55:00,690 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,690 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,692 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,692 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:00,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:00,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:00,692 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:00,692 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:00,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:55:00,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:00,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:55:00,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:00,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:55:00,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:55:00,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:00,699 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:00,699 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:55:00,699 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:55:01,119 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010360e7b8b56efb46d48c2a615541f9099e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010360e7b8b56efb46d48c2a615541f9099e 2023-03-09 01:55:01,524 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/a07fa68ebd004a50b35e5430955f0fa1 2023-03-09 01:55:01,529 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/a07fa68ebd004a50b35e5430955f0fa1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a07fa68ebd004a50b35e5430955f0fa1 2023-03-09 01:55:01,532 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a07fa68ebd004a50b35e5430955f0fa1, entries=1, sequenceid=33, filesize=5.0 K 2023-03-09 01:55:01,533 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 834ms, sequenceid=33, compaction requested=true 2023-03-09 01:55:01,533 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:01,533 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:55:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:55:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:55:01,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:55:01,535 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,535 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:01,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:01,535 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,535 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:01,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:01,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:01,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:01,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:55:01,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e16b322[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:01,536 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:55:01,536 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,538 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,538 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:55:01,538 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:55:01,538 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:01,538 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,538 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:01,540 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:55:01,540 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:01,540 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,541 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:55:01,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:01,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:01,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:01,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:01,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:01,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:01,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:55:01,543 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:55:01,543 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:01,543 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:01,543 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,543 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:01,545 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,545 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:01,545 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,545 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:01,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:01,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,546 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:01,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,548 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:01,548 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,548 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,548 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,548 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:55:01,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:01,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:01,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:01,548 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:01,548 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,548 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:01,549 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:01,549 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-09 01:55:01,549 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,549 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:01,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:01,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:01,649 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:01,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:01,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:01,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:01,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,656 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:55:01,656 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:01,656 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:55:01,656 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:55:01,656 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,656 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,659 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:01,659 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:01,659 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:01,659 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:01,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,660 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:55:01,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:55:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:55:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:55:01,660 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:01,660 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:01,660 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:55:01,660 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:55:02,079 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079c6768aac69546e8aed44209e4177b8b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509079c6768aac69546e8aed44209e4177b8b 2023-03-09 01:55:02,484 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/8c4ed25254bc4cc38d7bc6c3d22ba854 2023-03-09 01:55:02,489 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/8c4ed25254bc4cc38d7bc6c3d22ba854 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/8c4ed25254bc4cc38d7bc6c3d22ba854 2023-03-09 01:55:02,493 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/8c4ed25254bc4cc38d7bc6c3d22ba854, entries=1, sequenceid=37, filesize=5.0 K 2023-03-09 01:55:02,494 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 834ms, sequenceid=37, compaction requested=true 2023-03-09 01:55:02,494 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:02,494 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:02,494 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:55:02,494 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:55:02,494 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,494 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:55:02,494 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:55:02,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,496 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:02,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:02,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:02,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:02,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:02,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:02,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:55:02,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7be82875[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:02,497 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:55:02,497 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,499 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:55:02,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:55:02,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:02,499 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,499 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:02,500 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,500 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:55:02,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:02,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:02,500 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:02,501 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:02,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:02,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:55:02,502 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:55:02,503 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:02,503 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:02,503 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,503 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:02,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:02,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:02,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:02,505 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,506 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:02,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:02,509 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:55:02,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:02,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:02,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:02,509 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:02,610 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:02,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:02,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:02,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:02,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,616 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:55:02,616 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:02,616 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-09 01:55:02,616 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-09 01:55:02,616 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,616 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,618 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,618 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:02,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:02,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:02,618 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:02,618 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:02,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:02,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,619 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,619 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-09 01:55:02,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:02,619 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-09 01:55:02,619 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:02,619 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-09 01:55:02,619 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:55:02,619 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:02,619 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:02,619 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. started... 2023-03-09 01:55:02,620 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 527ef65d68bd3fd8127836853154b8b0 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:55:03,031 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511204a78688d12a54b9794111704e7581880 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204a78688d12a54b9794111704e7581880 2023-03-09 01:55:03,437 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/fb877f06c4454796bde55bddaeeaa6b6 2023-03-09 01:55:03,443 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/fb877f06c4454796bde55bddaeeaa6b6 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fb877f06c4454796bde55bddaeeaa6b6 2023-03-09 01:55:03,447 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fb877f06c4454796bde55bddaeeaa6b6, entries=1, sequenceid=41, filesize=5.0 K 2023-03-09 01:55:03,448 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 527ef65d68bd3fd8127836853154b8b0 in 828ms, sequenceid=41, compaction requested=true 2023-03-09 01:55:03,449 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:03,449 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:03,449 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:55:03,449 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:55:03,449 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,449 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-09 01:55:03,449 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-09 01:55:03,451 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,451 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:03,451 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,451 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:03,453 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-09 01:55:03,453 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d0a7b15[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:03,453 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-09 01:55:03,453 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,455 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,455 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-09 01:55:03,455 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-09 01:55:03,455 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:03,455 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,455 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:03,456 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,456 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-09 01:55:03,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:03,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:03,456 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:03,456 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-09 01:55:03,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:03,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:03,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:03,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:03,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-09 01:55:03,458 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-09 01:55:03,458 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:03,458 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:03,458 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,458 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:03,460 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,460 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:03,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:03,460 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:03,460 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:03,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:03,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:03,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:03,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,461 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:03,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,463 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,463 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:03,463 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:03,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-09 01:55:03,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:03,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:03,463 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:03,463 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:03,464 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,464 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:03,464 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:03,464 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:03,464 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-09 01:55:03,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:03,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:03,464 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,464 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,464 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,564 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:03,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:03,566 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:03,567 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:03,568 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:03,569 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327503569 2023-03-09 01:55:03,569 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:55:03,572 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:55:03,573 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-09 01:55:03,573 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:03,574 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x3cff5645 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:03,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ab02918, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:03,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,429 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010332b64f811be64c65978d1b604dad768d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010332b64f811be64c65978d1b604dad768d 2023-03-09 01:55:04,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:04,442 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:55:04,443 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35814, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:55:04,452 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:55:04,453 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35034, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:55:04,462 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7daac49abf2a445c842b043de6575366 first=Optional[r3] last=Optional[r6] 2023-03-09 01:55:04,463 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7daac49abf2a445c842b043de6575366}] 2023-03-09 01:55:04,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7daac49abf2a445c842b043de6575366 for inclusion in 527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:04,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-09 01:55:04,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:55:04,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:04,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7daac49abf2a445c842b043de6575366 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__oaarun3htgl8oqe4hmbam0k8e94naofi9kktiivv82v0erut2sgbcrh6bmc6oha3/family1/7daac49abf2a445c842b043de6575366 2023-03-09 01:55:04,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__oaarun3htgl8oqe4hmbam0k8e94naofi9kktiivv82v0erut2sgbcrh6bmc6oha3/family1/7daac49abf2a445c842b043de6575366 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_ 2023-03-09 01:55:04,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__oaarun3htgl8oqe4hmbam0k8e94naofi9kktiivv82v0erut2sgbcrh6bmc6oha3/family1/7daac49abf2a445c842b043de6575366 into 527ef65d68bd3fd8127836853154b8b0/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_ - updating store file list. 2023-03-09 01:55:04,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_ into 527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:04,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__oaarun3htgl8oqe4hmbam0k8e94naofi9kktiivv82v0erut2sgbcrh6bmc6oha3/family1/7daac49abf2a445c842b043de6575366 into 527ef65d68bd3fd8127836853154b8b0/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_) 2023-03-09 01:55:04,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__oaarun3htgl8oqe4hmbam0k8e94naofi9kktiivv82v0erut2sgbcrh6bmc6oha3/family1/7daac49abf2a445c842b043de6575366 2023-03-09 01:55:04,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-09 01:55:04,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-09 01:55:04,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 527ef65d68bd3fd8127836853154b8b0/family1 is initiating minor compaction 2023-03-09 01:55:04,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6d9693c9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:04,481 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 527ef65d68bd3fd8127836853154b8b0/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:04,481 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/bd7888532fb2471389b418ca6fca85d8, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/afd59a9090274183ba7706ba31728eaf, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e7afa8377786468f9e09ed221cd2e155, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/852aae59f1804b6188ae8a67410f4374, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fc6e0c31b4b5475aa430a76fd9768735, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/b70eccaa49e94878a98e212093d7ea72, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a07fa68ebd004a50b35e5430955f0fa1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/8c4ed25254bc4cc38d7bc6c3d22ba854, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fb877f06c4454796bde55bddaeeaa6b6, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp, totalSize=50.3 K 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting bd7888532fb2471389b418ca6fca85d8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting afd59a9090274183ba7706ba31728eaf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting e7afa8377786468f9e09ed221cd2e155, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 852aae59f1804b6188ae8a67410f4374, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting fc6e0c31b4b5475aa430a76fd9768735, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting b70eccaa49e94878a98e212093d7ea72, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting a07fa68ebd004a50b35e5430955f0fa1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 8c4ed25254bc4cc38d7bc6c3d22ba854, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-09 01:55:04,481 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting fb877f06c4454796bde55bddaeeaa6b6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-09 01:55:04,482 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-09 01:55:04,483 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:04,483 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:04,485 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d310e94d9b464fdbbaad3f4783fe76aa to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d310e94d9b464fdbbaad3f4783fe76aa 2023-03-09 01:55:04,486 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512319371952b8c3642be996f0c7f4ac0e263 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512319371952b8c3642be996f0c7f4ac0e263 2023-03-09 01:55:04,488 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010159dbc52de55e4259896213d5ad166384 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010159dbc52de55e4259896213d5ad166384 2023-03-09 01:55:04,489 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010360e7b8b56efb46d48c2a615541f9099e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010360e7b8b56efb46d48c2a615541f9099e 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-09 01:55:04,489 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:04,547 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 527ef65d68bd3fd8127836853154b8b0#family1#compaction#72 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:55:04,983 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/7d83e8a440d44306b070d74daf3c6da4 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/7d83e8a440d44306b070d74daf3c6da4 2023-03-09 01:55:04,987 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 527ef65d68bd3fd8127836853154b8b0/family1 of 527ef65d68bd3fd8127836853154b8b0 into 7d83e8a440d44306b070d74daf3c6da4(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:55:04,987 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:04,988 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1678326904481; duration=0sec 2023-03-09 01:55:04,988 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:05,023 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/bd7888532fb2471389b418ca6fca85d8, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/afd59a9090274183ba7706ba31728eaf, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e7afa8377786468f9e09ed221cd2e155, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/852aae59f1804b6188ae8a67410f4374, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fc6e0c31b4b5475aa430a76fd9768735, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/b70eccaa49e94878a98e212093d7ea72, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a07fa68ebd004a50b35e5430955f0fa1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/8c4ed25254bc4cc38d7bc6c3d22ba854, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fb877f06c4454796bde55bddaeeaa6b6, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_] to archive 2023-03-09 01:55:05,024 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:05,026 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/bd7888532fb2471389b418ca6fca85d8 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/bd7888532fb2471389b418ca6fca85d8 2023-03-09 01:55:05,027 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/afd59a9090274183ba7706ba31728eaf to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/afd59a9090274183ba7706ba31728eaf 2023-03-09 01:55:05,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e7afa8377786468f9e09ed221cd2e155 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e7afa8377786468f9e09ed221cd2e155 2023-03-09 01:55:05,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/852aae59f1804b6188ae8a67410f4374 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/852aae59f1804b6188ae8a67410f4374 2023-03-09 01:55:05,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fc6e0c31b4b5475aa430a76fd9768735 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fc6e0c31b4b5475aa430a76fd9768735 2023-03-09 01:55:05,031 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/b70eccaa49e94878a98e212093d7ea72 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/b70eccaa49e94878a98e212093d7ea72 2023-03-09 01:55:05,033 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a07fa68ebd004a50b35e5430955f0fa1 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a07fa68ebd004a50b35e5430955f0fa1 2023-03-09 01:55:05,034 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/8c4ed25254bc4cc38d7bc6c3d22ba854 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/8c4ed25254bc4cc38d7bc6c3d22ba854 2023-03-09 01:55:05,035 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fb877f06c4454796bde55bddaeeaa6b6 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/fb877f06c4454796bde55bddaeeaa6b6 2023-03-09 01:55:05,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a4e3ec4546354c679f7481f2bc35ad6e_SeqId_44_ 2023-03-09 01:55:05,379 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ca972f2350d645ca861004bcfcf65f2f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205ca972f2350d645ca861004bcfcf65f2f 2023-03-09 01:55:05,379 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:05,404 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5758e7cc073f4b84a3d06aa235677260 first=Optional[r00] last=Optional[r2] 2023-03-09 01:55:05,405 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5758e7cc073f4b84a3d06aa235677260}] 2023-03-09 01:55:05,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5758e7cc073f4b84a3d06aa235677260 for inclusion in 527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:05,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-09 01:55:05,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:55:05,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:05,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5758e7cc073f4b84a3d06aa235677260 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cp8agpvjm3q7413a8gdpguqv7jtpjq96bhlrcvaj1alsibh91cc31cq6fleqduqm/family1/5758e7cc073f4b84a3d06aa235677260 2023-03-09 01:55:05,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cp8agpvjm3q7413a8gdpguqv7jtpjq96bhlrcvaj1alsibh91cc31cq6fleqduqm/family1/5758e7cc073f4b84a3d06aa235677260 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a1ee628a451c432a882f866296df6d7f_SeqId_47_ 2023-03-09 01:55:05,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cp8agpvjm3q7413a8gdpguqv7jtpjq96bhlrcvaj1alsibh91cc31cq6fleqduqm/family1/5758e7cc073f4b84a3d06aa235677260 into 527ef65d68bd3fd8127836853154b8b0/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a1ee628a451c432a882f866296df6d7f_SeqId_47_ - updating store file list. 2023-03-09 01:55:05,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a1ee628a451c432a882f866296df6d7f_SeqId_47_ into 527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:05,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cp8agpvjm3q7413a8gdpguqv7jtpjq96bhlrcvaj1alsibh91cc31cq6fleqduqm/family1/5758e7cc073f4b84a3d06aa235677260 into 527ef65d68bd3fd8127836853154b8b0/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a1ee628a451c432a882f866296df6d7f_SeqId_47_) 2023-03-09 01:55:05,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cp8agpvjm3q7413a8gdpguqv7jtpjq96bhlrcvaj1alsibh91cc31cq6fleqduqm/family1/5758e7cc073f4b84a3d06aa235677260 2023-03-09 01:55:05,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-09 01:55:05,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-09 01:55:05,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 527ef65d68bd3fd8127836853154b8b0/family1 is initiating minor compaction (all files) 2023-03-09 01:55:05,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@196a3a18; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:05,421 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 527ef65d68bd3fd8127836853154b8b0/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:05,421 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e99ae4d851cf48c5b0dc5e7889b7a7a1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/7d83e8a440d44306b070d74daf3c6da4, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a1ee628a451c432a882f866296df6d7f_SeqId_47_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp, totalSize=388.1 K 2023-03-09 01:55:05,422 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting e99ae4d851cf48c5b0dc5e7889b7a7a1, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843359 2023-03-09 01:55:05,422 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 7d83e8a440d44306b070d74daf3c6da4, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243359 2023-03-09 01:55:05,423 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting a1ee628a451c432a882f866296df6d7f_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878843359 2023-03-09 01:55:05,423 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:05,423 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:05,425 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113029606e22716248c49f52bed13d44452f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113029606e22716248c49f52bed13d44452f 2023-03-09 01:55:05,426 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120153cd58d125ca48e286133ef7f2091062 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120153cd58d125ca48e286133ef7f2091062 2023-03-09 01:55:05,427 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205c82e3f8bd0de4020b137e73d254c7604 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205c82e3f8bd0de4020b137e73d254c7604 2023-03-09 01:55:05,427 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-09 01:55:05,427 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-09 01:55:05,428 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:05,428 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:05,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:55:05,430 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:05,432 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8630 sec 2023-03-09 01:55:05,443 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 527ef65d68bd3fd8127836853154b8b0#family1#compaction#73 average throughput is 16.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:55:05,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-09 01:55:05,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:05,910 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-09 01:55:05,911 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-09 01:55:05,911 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-09 01:55:05,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-09 01:55:05,915 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, REOPEN/MOVE}] 2023-03-09 01:55:05,916 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, REOPEN/MOVE 2023-03-09 01:55:05,916 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:05,917 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326905916"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326905916"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326905916"}]},"ts":"1678326905916"} 2023-03-09 01:55:05,918 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:06,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:06,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 527ef65d68bd3fd8127836853154b8b0, disabling compactions & flushes 2023-03-09 01:55:06,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:06,662 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/.tmp/family1/f32c6e287a074e85b93a8d46474b0164 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/f32c6e287a074e85b93a8d46474b0164 2023-03-09 01:55:06,667 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 527ef65d68bd3fd8127836853154b8b0/family1 of 527ef65d68bd3fd8127836853154b8b0 into f32c6e287a074e85b93a8d46474b0164(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:06,667 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:06,667 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:06,667 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1678326905421; duration=1sec 2023-03-09 01:55:06,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:06,667 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:06,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. after waiting 0 ms 2023-03-09 01:55:06,667 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:06,670 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e99ae4d851cf48c5b0dc5e7889b7a7a1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/7d83e8a440d44306b070d74daf3c6da4, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a1ee628a451c432a882f866296df6d7f_SeqId_47_] to archive 2023-03-09 01:55:06,671 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:06,673 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e99ae4d851cf48c5b0dc5e7889b7a7a1 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/e99ae4d851cf48c5b0dc5e7889b7a7a1 2023-03-09 01:55:06,674 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/7d83e8a440d44306b070d74daf3c6da4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/7d83e8a440d44306b070d74daf3c6da4 2023-03-09 01:55:06,675 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a1ee628a451c432a882f866296df6d7f_SeqId_47_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/a1ee628a451c432a882f866296df6d7f_SeqId_47_ 2023-03-09 01:55:06,679 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-09 01:55:06,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:06,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:06,680 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 527ef65d68bd3fd8127836853154b8b0 to self. 2023-03-09 01:55:06,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:06,682 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=CLOSED 2023-03-09 01:55:06,682 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326906682"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326906682"}]},"ts":"1678326906682"} 2023-03-09 01:55:06,684 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-09 01:55:06,685 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757 in 765 msec 2023-03-09 01:55:06,685 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=true 2023-03-09 01:55:06,835 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:06,836 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326906835"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326906835"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326906835"}]},"ts":"1678326906835"} 2023-03-09 01:55:06,837 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:06,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:06,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 527ef65d68bd3fd8127836853154b8b0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:55:06,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:06,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:06,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:06,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:06,994 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:06,994 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:06,994 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:06,995 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 527ef65d68bd3fd8127836853154b8b0 columnFamilyName family1 2023-03-09 01:55:06,999 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/f32c6e287a074e85b93a8d46474b0164 2023-03-09 01:55:06,999 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(310): Store=527ef65d68bd3fd8127836853154b8b0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:07,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:07,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:07,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:07,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 527ef65d68bd3fd8127836853154b8b0; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67832082, jitterRate=0.010776787996292114}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:55:07,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:07,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0., pid=63, masterSystemTime=1678326906989 2023-03-09 01:55:07,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:07,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:07,007 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:07,008 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326907007"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326907007"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326907007"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326907007"}]},"ts":"1678326907007"} 2023-03-09 01:55:07,010 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-09 01:55:07,010 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757 in 172 msec 2023-03-09 01:55:07,011 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-09 01:55:07,012 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, REOPEN/MOVE in 1.0950 sec 2023-03-09 01:55:07,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-09 01:55:07,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0990 sec 2023-03-09 01:55:07,015 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5150 sec 2023-03-09 01:55:07,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-09 01:55:07,017 INFO [Listener at localhost/33973] compactions.TestMobCompactor(278): alter status finished 2023-03-09 01:55:07,018 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:07,019 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:07,020 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:07,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:55:07,020 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327507020 2023-03-09 01:55:07,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:55:07,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-09 01:55:07,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:07,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x5263d421 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:07,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e41ceb9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:07,027 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:07,027 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:07,027 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:07,027 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:07,028 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:07,852 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112856c2e50e662c464083c70afbbf27013a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112856c2e50e662c464083c70afbbf27013a 2023-03-09 01:55:07,853 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:07,865 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:55:07,866 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35820, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:55:07,872 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:55:07,874 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35044, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:55:07,882 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/64c3ec24d3844a8d9ac3a36879a8edac first=Optional[r05] last=Optional[r8] 2023-03-09 01:55:07,883 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/64c3ec24d3844a8d9ac3a36879a8edac}] 2023-03-09 01:55:07,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/64c3ec24d3844a8d9ac3a36879a8edac for inclusion in 527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:07,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-09 01:55:07,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:55:07,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:07,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/64c3ec24d3844a8d9ac3a36879a8edac to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r2n4q0mrsaj9vcve1vib1n2jkp21m3j4u4pcq7d1nqi41ka553e0onev404frusp/family1/64c3ec24d3844a8d9ac3a36879a8edac 2023-03-09 01:55:07,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r2n4q0mrsaj9vcve1vib1n2jkp21m3j4u4pcq7d1nqi41ka553e0onev404frusp/family1/64c3ec24d3844a8d9ac3a36879a8edac as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/f314815aab0b41df8ea19d0b6fe6d60f_SeqId_53_ 2023-03-09 01:55:07,893 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r2n4q0mrsaj9vcve1vib1n2jkp21m3j4u4pcq7d1nqi41ka553e0onev404frusp/family1/64c3ec24d3844a8d9ac3a36879a8edac into 527ef65d68bd3fd8127836853154b8b0/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/f314815aab0b41df8ea19d0b6fe6d60f_SeqId_53_ - updating store file list. 2023-03-09 01:55:07,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/f314815aab0b41df8ea19d0b6fe6d60f_SeqId_53_ into 527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:07,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r2n4q0mrsaj9vcve1vib1n2jkp21m3j4u4pcq7d1nqi41ka553e0onev404frusp/family1/64c3ec24d3844a8d9ac3a36879a8edac into 527ef65d68bd3fd8127836853154b8b0/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/f314815aab0b41df8ea19d0b6fe6d60f_SeqId_53_) 2023-03-09 01:55:07,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r2n4q0mrsaj9vcve1vib1n2jkp21m3j4u4pcq7d1nqi41ka553e0onev404frusp/family1/64c3ec24d3844a8d9ac3a36879a8edac 2023-03-09 01:55:07,899 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:07,900 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:07,901 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204a78688d12a54b9794111704e7581880 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204a78688d12a54b9794111704e7581880 2023-03-09 01:55:07,902 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288be8f660d096418b9b321aba56b61d7c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288be8f660d096418b9b321aba56b61d7c 2023-03-09 01:55:07,902 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-09 01:55:07,902 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-09 01:55:07,903 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:07,903 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:07,903 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:07,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:07,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:55:07,904 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:07,905 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 886 msec 2023-03-09 01:55:07,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-09 01:55:07,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:08,368 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-09 01:55:08,369 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-09 01:55:08,369 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-09 01:55:08,371 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-09 01:55:08,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, REOPEN/MOVE}] 2023-03-09 01:55:08,374 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, REOPEN/MOVE 2023-03-09 01:55:08,374 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:08,375 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326908374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326908374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326908374"}]},"ts":"1678326908374"} 2023-03-09 01:55:08,376 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:08,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 527ef65d68bd3fd8127836853154b8b0, disabling compactions & flushes 2023-03-09 01:55:08,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:08,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:08,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. after waiting 0 ms 2023-03-09 01:55:08,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:08,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-09 01:55:08,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:08,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:08,534 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 527ef65d68bd3fd8127836853154b8b0 to self. 2023-03-09 01:55:08,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,536 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=CLOSED 2023-03-09 01:55:08,536 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326908536"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326908536"}]},"ts":"1678326908536"} 2023-03-09 01:55:08,538 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-09 01:55:08,538 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757 in 161 msec 2023-03-09 01:55:08,538 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=true 2023-03-09 01:55:08,689 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:08,689 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326908689"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326908689"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326908689"}]},"ts":"1678326908689"} 2023-03-09 01:55:08,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:08,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:08,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 527ef65d68bd3fd8127836853154b8b0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:55:08,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:08,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,846 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,847 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:08,847 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1 2023-03-09 01:55:08,848 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 527ef65d68bd3fd8127836853154b8b0 columnFamilyName family1 2023-03-09 01:55:08,853 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/f314815aab0b41df8ea19d0b6fe6d60f_SeqId_53_ 2023-03-09 01:55:08,855 DEBUG [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/family1/f32c6e287a074e85b93a8d46474b0164 2023-03-09 01:55:08,855 INFO [StoreOpener-527ef65d68bd3fd8127836853154b8b0-1] regionserver.HStore(310): Store=527ef65d68bd3fd8127836853154b8b0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:08,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:55:08,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 527ef65d68bd3fd8127836853154b8b0; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69374874, jitterRate=0.03376618027687073}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:55:08,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:55:08,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0., pid=69, masterSystemTime=1678326908842 2023-03-09 01:55:08,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:08,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:55:08,862 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=527ef65d68bd3fd8127836853154b8b0, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:08,862 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1678326908862"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326908862"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326908862"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326908862"}]},"ts":"1678326908862"} 2023-03-09 01:55:08,864 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-09 01:55:08,864 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 527ef65d68bd3fd8127836853154b8b0, server=jenkins-hbase4.apache.org,46511,1678326819757 in 173 msec 2023-03-09 01:55:08,865 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-09 01:55:08,866 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=527ef65d68bd3fd8127836853154b8b0, REOPEN/MOVE in 491 msec 2023-03-09 01:55:08,867 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-09 01:55:08,867 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 495 msec 2023-03-09 01:55:08,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 910 msec 2023-03-09 01:55:08,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-09 01:55:08,870 INFO [Listener at localhost/33973] compactions.TestMobCompactor(278): alter status finished 2023-03-09 01:55:08,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-09 01:55:08,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:08,872 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:08,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:55:08,873 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327508873 2023-03-09 01:55:08,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:55:08,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-09 01:55:08,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:08,875 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x1597f568 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:08,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2478395, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:08,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:08,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:08,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:08,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:08,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:08,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:08,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:08,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-09 01:55:08,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-09 01:55:08,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:55:08,880 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-09 01:55:08,881 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 9 msec 2023-03-09 01:55:08,926 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=246 (was 237) - Thread LEAK? -, OpenFileDescriptor=630 (was 596) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=157 (was 173), ProcessCount=171 (was 171), AvailableMemoryMB=8087 (was 8092) 2023-03-09 01:55:08,938 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=246, OpenFileDescriptor=630, MaxFileDescriptor=60000, SystemLoadAverage=157, ProcessCount=171, AvailableMemoryMB=8086 2023-03-09 01:55:08,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-09 01:55:08,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-09 01:55:08,945 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:55:08,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-09 01:55:08,945 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:55:08,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-09 01:55:08,946 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:08,947 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6 empty. 2023-03-09 01:55:08,947 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:08,947 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-09 01:55:09,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-09 01:55:09,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-09 01:55:09,355 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-09 01:55:09,357 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4a041e81b0e6a47b9bdeb6ca8e7698c6, NAME => 'testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:55:09,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-09 01:55:09,762 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:09,762 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 4a041e81b0e6a47b9bdeb6ca8e7698c6, disabling compactions & flushes 2023-03-09 01:55:09,762 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:09,762 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:09,762 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. after waiting 0 ms 2023-03-09 01:55:09,762 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:09,762 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:09,762 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 4a041e81b0e6a47b9bdeb6ca8e7698c6: 2023-03-09 01:55:09,765 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:55:09,765 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678326909765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326909765"}]},"ts":"1678326909765"} 2023-03-09 01:55:09,767 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:55:09,767 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:55:09,768 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326909768"}]},"ts":"1678326909768"} 2023-03-09 01:55:09,768 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-09 01:55:09,772 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=4a041e81b0e6a47b9bdeb6ca8e7698c6, ASSIGN}] 2023-03-09 01:55:09,774 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=4a041e81b0e6a47b9bdeb6ca8e7698c6, ASSIGN 2023-03-09 01:55:09,774 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=4a041e81b0e6a47b9bdeb6ca8e7698c6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:55:09,925 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=4a041e81b0e6a47b9bdeb6ca8e7698c6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:09,926 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678326909925"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326909925"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326909925"}]},"ts":"1678326909925"} 2023-03-09 01:55:09,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 4a041e81b0e6a47b9bdeb6ca8e7698c6, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:10,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-09 01:55:10,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:10,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4a041e81b0e6a47b9bdeb6ca8e7698c6, NAME => 'testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:55:10,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-09 01:55:10,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-09 01:55:10,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:10,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:10,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:10,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:10,086 INFO [StoreOpener-4a041e81b0e6a47b9bdeb6ca8e7698c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:10,087 DEBUG [StoreOpener-4a041e81b0e6a47b9bdeb6ca8e7698c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 2023-03-09 01:55:10,087 DEBUG [StoreOpener-4a041e81b0e6a47b9bdeb6ca8e7698c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 2023-03-09 01:55:10,088 INFO [StoreOpener-4a041e81b0e6a47b9bdeb6ca8e7698c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4a041e81b0e6a47b9bdeb6ca8e7698c6 columnFamilyName f1 2023-03-09 01:55:10,089 INFO [StoreOpener-4a041e81b0e6a47b9bdeb6ca8e7698c6-1] regionserver.HStore(310): Store=4a041e81b0e6a47b9bdeb6ca8e7698c6/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:10,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:10,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:10,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:10,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:55:10,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4a041e81b0e6a47b9bdeb6ca8e7698c6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71690365, jitterRate=0.06826968491077423}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:55:10,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4a041e81b0e6a47b9bdeb6ca8e7698c6: 2023-03-09 01:55:10,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6., pid=73, masterSystemTime=1678326910079 2023-03-09 01:55:10,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:10,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:10,097 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=4a041e81b0e6a47b9bdeb6ca8e7698c6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:10,097 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678326910097"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326910097"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326910097"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326910097"}]},"ts":"1678326910097"} 2023-03-09 01:55:10,099 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-09 01:55:10,099 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 4a041e81b0e6a47b9bdeb6ca8e7698c6, server=jenkins-hbase4.apache.org,46511,1678326819757 in 171 msec 2023-03-09 01:55:10,101 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-09 01:55:10,101 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=4a041e81b0e6a47b9bdeb6ca8e7698c6, ASSIGN in 327 msec 2023-03-09 01:55:10,101 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:55:10,101 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326910101"}]},"ts":"1678326910101"} 2023-03-09 01:55:10,102 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-09 01:55:10,104 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:55:10,105 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1610 sec 2023-03-09 01:55:11,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-09 01:55:11,051 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-09 01:55:11,051 DEBUG [Listener at localhost/33973] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-09 01:55:11,053 INFO [Listener at localhost/33973] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-09 01:55:11,056 INFO [Listener at localhost/33973] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-09 01:55:11,057 INFO [Listener at localhost/33973] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-09 01:55:11,057 INFO [Listener at localhost/33973] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-09 01:55:11,061 INFO [Listener at localhost/33973] regionserver.HRegion(2745): Flushing 4a041e81b0e6a47b9bdeb6ca8e7698c6 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-09 01:55:11,473 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309829a2b34a3eb4a41a0b2ea1b27437f03 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309829a2b34a3eb4a41a0b2ea1b27437f03 2023-03-09 01:55:11,558 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-09 01:55:11,878 INFO [Listener at localhost/33973] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/.tmp/f1/2b7db58540b0442ca2ead3ae7274e1e4 2023-03-09 01:55:11,883 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/.tmp/f1/2b7db58540b0442ca2ead3ae7274e1e4 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4 2023-03-09 01:55:11,887 INFO [Listener at localhost/33973] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4, entries=1, sequenceid=5, filesize=4.9 K 2023-03-09 01:55:11,888 INFO [Listener at localhost/33973] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 4a041e81b0e6a47b9bdeb6ca8e7698c6 in 827ms, sequenceid=5, compaction requested=false 2023-03-09 01:55:11,889 DEBUG [Listener at localhost/33973] regionserver.HRegion(2446): Flush status journal for 4a041e81b0e6a47b9bdeb6ca8e7698c6: 2023-03-09 01:55:11,891 INFO [Listener at localhost/33973] regionserver.HRegion(2745): Flushing 4a041e81b0e6a47b9bdeb6ca8e7698c6 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-09 01:55:12,218 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:55:12,303 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309d8b99888ef4942919eeeb6bed84cfe6f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309d8b99888ef4942919eeeb6bed84cfe6f 2023-03-09 01:55:12,716 INFO [Listener at localhost/33973] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/.tmp/f1/5efe2fc6ae594a089ee677f763ee23ab 2023-03-09 01:55:12,720 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/.tmp/f1/5efe2fc6ae594a089ee677f763ee23ab as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab 2023-03-09 01:55:12,724 INFO [Listener at localhost/33973] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab, entries=1, sequenceid=9, filesize=4.9 K 2023-03-09 01:55:12,725 INFO [Listener at localhost/33973] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 4a041e81b0e6a47b9bdeb6ca8e7698c6 in 834ms, sequenceid=9, compaction requested=false 2023-03-09 01:55:12,725 DEBUG [Listener at localhost/33973] regionserver.HRegion(2446): Flush status journal for 4a041e81b0e6a47b9bdeb6ca8e7698c6: 2023-03-09 01:55:12,726 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-09 01:55:12,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-09 01:55:12,729 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-09 01:55:12,729 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:55:12,729 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327512729 2023-03-09 01:55:12,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-09 01:55:12,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-09 01:55:12,730 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-09 01:55:12,731 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x492fa0d3 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:12,735 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77cac948, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:12,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230309 for table='testGetAfterCompaction' and column='f1' 2023-03-09 01:55:12,778 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,778 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,829 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,829 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,880 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,880 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,931 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,931 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,982 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:12,982 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,033 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,033 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,084 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,084 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,134 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,135 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,158 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309e23dcfdc2d934baa87942d3d7b7c7c00 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309e23dcfdc2d934baa87942d3d7b7c7c00 2023-03-09 01:55:13,159 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='testGetAfterCompaction' column='f1' 2023-03-09 01:55:13,176 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:55:13,177 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35830, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:55:13,184 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:55:13,186 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35060, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:55:13,186 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,186 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,194 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230309/f1/dac52e46bb1344e4afdb5451cdb676f7 first=Optional[r1] last=Optional[r2] 2023-03-09 01:55:13,195 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{f1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230309/f1/dac52e46bb1344e4afdb5451cdb676f7}] 2023-03-09 01:55:13,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230309/f1/dac52e46bb1344e4afdb5451cdb676f7 for inclusion in 4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 2023-03-09 01:55:13,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-09 01:55:13,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:55:13,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 4a041e81b0e6a47b9bdeb6ca8e7698c6: 2023-03-09 01:55:13,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230309/f1/dac52e46bb1344e4afdb5451cdb676f7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testGetAfterCompaction__egvajj2gumml459donomj8v8j122vhilhat7s9582bnkus5bbmppoha40dqi0tl0/f1/dac52e46bb1344e4afdb5451cdb676f7 2023-03-09 01:55:13,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testGetAfterCompaction__egvajj2gumml459donomj8v8j122vhilhat7s9582bnkus5bbmppoha40dqi0tl0/f1/dac52e46bb1344e4afdb5451cdb676f7 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_ 2023-03-09 01:55:13,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testGetAfterCompaction__egvajj2gumml459donomj8v8j122vhilhat7s9582bnkus5bbmppoha40dqi0tl0/f1/dac52e46bb1344e4afdb5451cdb676f7 into 4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_ - updating store file list. 2023-03-09 01:55:13,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_ into 4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 2023-03-09 01:55:13,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testGetAfterCompaction__egvajj2gumml459donomj8v8j122vhilhat7s9582bnkus5bbmppoha40dqi0tl0/f1/dac52e46bb1344e4afdb5451cdb676f7 into 4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_) 2023-03-09 01:55:13,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testGetAfterCompaction__egvajj2gumml459donomj8v8j122vhilhat7s9582bnkus5bbmppoha40dqi0tl0/f1/dac52e46bb1344e4afdb5451cdb676f7 2023-03-09 01:55:13,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): 4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 is initiating minor compaction 2023-03-09 01:55:13,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7aca7de9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:13,212 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 in testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:13,212 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/.tmp, totalSize=10.0 K 2023-03-09 01:55:13,212 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 5efe2fc6ae594a089ee677f763ee23ab, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-09 01:55:13,212 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 3926c7e42b804599b8d4a967bf4327e3_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-09 01:55:13,214 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='testGetAfterCompaction' column='f1' 2023-03-09 01:55:13,215 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:13,217 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309829a2b34a3eb4a41a0b2ea1b27437f03 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309829a2b34a3eb4a41a0b2ea1b27437f03 2023-03-09 01:55:13,218 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309d8b99888ef4942919eeeb6bed84cfe6f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309d8b99888ef4942919eeeb6bed84cfe6f 2023-03-09 01:55:13,218 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-09 01:55:13,218 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230309' 2023-03-09 01:55:13,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230309 for table='testGetAfterCompaction' and column='f1' 2023-03-09 01:55:13,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-09 01:55:13,219 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4a041e81b0e6a47b9bdeb6ca8e7698c6#f1#compaction#76 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:55:13,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:55:13,220 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-09 01:55:13,222 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 493 msec 2023-03-09 01:55:13,237 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,237 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,237 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,242 DEBUG [Listener at localhost/33973] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-09 01:55:13,246 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35836, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-09 01:55:13,246 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46511] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:13,247 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-09 01:55:13,247 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-09 01:55:13,247 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46511] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-09 01:55:13,247 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46511] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. because compaction request was cancelled 2023-03-09 01:55:13,299 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,299 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,299 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,350 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,350 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,350 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,401 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,401 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,401 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,452 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,453 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,453 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,503 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,504 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,504 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,554 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,555 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,555 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,605 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,605 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,605 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,656 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,657 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,657 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,707 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,708 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,708 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,758 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,759 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,759 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,809 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,810 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,810 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,860 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,861 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,861 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,911 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,912 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,912 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,962 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:13,963 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:13,963 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:14,013 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:14,014 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:14,014 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:14,035 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/.tmp/f1/21691daf054444e488e54dd2a38310fb as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/21691daf054444e488e54dd2a38310fb 2023-03-09 01:55:14,040 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 4a041e81b0e6a47b9bdeb6ca8e7698c6/f1 of 4a041e81b0e6a47b9bdeb6ca8e7698c6 into 21691daf054444e488e54dd2a38310fb(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:55:14,040 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4a041e81b0e6a47b9bdeb6ca8e7698c6: 2023-03-09 01:55:14,040 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1678326913212; duration=0sec 2023-03-09 01:55:14,040 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:14,064 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/21691daf054444e488e54dd2a38310fb; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678326914030; access_time=1678326913217; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:14,065 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:14,065 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1678326913154; access_time=1678326912747; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-09 01:55:14,065 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326912715; access_time=1678326911893; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:14,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_] to archive 2023-03-09 01:55:14,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:14,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/5efe2fc6ae594a089ee677f763ee23ab 2023-03-09 01:55:14,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/3926c7e42b804599b8d4a967bf4327e3_SeqId_12_ 2023-03-09 01:55:14,115 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/21691daf054444e488e54dd2a38310fb; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1678326914030; access_time=1678326913217; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:14,115 INFO [Listener at localhost/33973] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1678326911878; access_time=1678326911063; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-09 01:55:14,233 INFO [Listener at localhost/33973] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-09 01:55:14,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-09 01:55:14,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-09 01:55:14,247 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326914247"}]},"ts":"1678326914247"} 2023-03-09 01:55:14,249 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-09 01:55:14,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-09 01:55:14,251 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-09 01:55:14,252 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=4a041e81b0e6a47b9bdeb6ca8e7698c6, UNASSIGN}] 2023-03-09 01:55:14,254 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=4a041e81b0e6a47b9bdeb6ca8e7698c6, UNASSIGN 2023-03-09 01:55:14,255 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=4a041e81b0e6a47b9bdeb6ca8e7698c6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:14,255 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678326914255"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326914255"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326914255"}]},"ts":"1678326914255"} 2023-03-09 01:55:14,257 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 4a041e81b0e6a47b9bdeb6ca8e7698c6, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:14,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-09 01:55:14,408 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:14,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4a041e81b0e6a47b9bdeb6ca8e7698c6, disabling compactions & flushes 2023-03-09 01:55:14,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:14,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:14,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. after waiting 0 ms 2023-03-09 01:55:14,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:14,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-09 01:55:14,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-09 01:55:14,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6. 2023-03-09 01:55:14,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4a041e81b0e6a47b9bdeb6ca8e7698c6: 2023-03-09 01:55:14,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:14,419 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=4a041e81b0e6a47b9bdeb6ca8e7698c6, regionState=CLOSED 2023-03-09 01:55:14,419 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1678326914419"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326914419"}]},"ts":"1678326914419"} 2023-03-09 01:55:14,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-09 01:55:14,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 4a041e81b0e6a47b9bdeb6ca8e7698c6, server=jenkins-hbase4.apache.org,46511,1678326819757 in 163 msec 2023-03-09 01:55:14,423 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-09 01:55:14,423 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=4a041e81b0e6a47b9bdeb6ca8e7698c6, UNASSIGN in 169 msec 2023-03-09 01:55:14,423 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326914423"}]},"ts":"1678326914423"} 2023-03-09 01:55:14,425 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-09 01:55:14,430 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-09 01:55:14,432 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 191 msec 2023-03-09 01:55:14,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-09 01:55:14,551 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-09 01:55:14,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-09 01:55:14,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-09 01:55:14,563 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-09 01:55:14,564 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-09 01:55:14,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-09 01:55:14,567 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:14,569 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1, FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/recovered.edits] 2023-03-09 01:55:14,573 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/21691daf054444e488e54dd2a38310fb to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/21691daf054444e488e54dd2a38310fb 2023-03-09 01:55:14,574 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/f1/2b7db58540b0442ca2ead3ae7274e1e4 2023-03-09 01:55:14,576 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/recovered.edits/15.seqid to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6/recovered.edits/15.seqid 2023-03-09 01:55:14,577 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testGetAfterCompaction/4a041e81b0e6a47b9bdeb6ca8e7698c6 2023-03-09 01:55:14,577 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-09 01:55:14,577 DEBUG [PEWorker-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-09 01:55:14,578 DEBUG [PEWorker-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-09 01:55:14,580 DEBUG [PEWorker-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309e23dcfdc2d934baa87942d3d7b7c7c00 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230309e23dcfdc2d934baa87942d3d7b7c7c00 2023-03-09 01:55:14,580 DEBUG [PEWorker-4] backup.HFileArchiver(596): Deleted hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-09 01:55:14,582 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-09 01:55:14,604 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-09 01:55:14,606 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-09 01:55:14,608 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-09 01:55:14,608 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-09 01:55:14,608 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678326914608"}]},"ts":"9223372036854775807"} 2023-03-09 01:55:14,609 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-09 01:55:14,609 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4a041e81b0e6a47b9bdeb6ca8e7698c6, NAME => 'testGetAfterCompaction,,1678326908942.4a041e81b0e6a47b9bdeb6ca8e7698c6.', STARTKEY => '', ENDKEY => ''}] 2023-03-09 01:55:14,610 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-09 01:55:14,610 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1678326914610"}]},"ts":"9223372036854775807"} 2023-03-09 01:55:14,611 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-09 01:55:14,613 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-09 01:55:14,614 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 57 msec 2023-03-09 01:55:14,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-09 01:55:14,667 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-09 01:55:14,680 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=250 (was 246) - Thread LEAK? -, OpenFileDescriptor=626 (was 630), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=152 (was 157), ProcessCount=171 (was 171), AvailableMemoryMB=8084 (was 8086) 2023-03-09 01:55:14,691 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=250, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=152, ProcessCount=171, AvailableMemoryMB=8083 2023-03-09 01:55:14,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-09 01:55:14,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-09 01:55:14,695 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:55:14,695 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-09 01:55:14,695 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:55:14,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-09 01:55:14,697 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:14,697 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:14,697 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:14,698 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e empty. 2023-03-09 01:55:14,698 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc empty. 2023-03-09 01:55:14,698 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1 empty. 2023-03-09 01:55:14,699 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:14,699 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:14,699 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:14,699 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-09 01:55:14,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-09 01:55:14,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-09 01:55:15,107 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-09 01:55:15,108 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 00f768e6a69add6056570cdcb3ad746e, NAME => 'testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:55:15,109 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => abbc0d228ef2735615e5e8f7e1c367fc, NAME => 'testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:55:15,109 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 18e9a9ff9da4b5322903ba24ba13b2e1, NAME => 'testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:55:15,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:15,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 00f768e6a69add6056570cdcb3ad746e, disabling compactions & flushes 2023-03-09 01:55:15,124 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:15,125 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:15,125 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. after waiting 0 ms 2023-03-09 01:55:15,125 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:15,125 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:15,125 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:15,127 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:15,127 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 18e9a9ff9da4b5322903ba24ba13b2e1, disabling compactions & flushes 2023-03-09 01:55:15,127 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:15,127 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:15,127 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. after waiting 0 ms 2023-03-09 01:55:15,127 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:15,127 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:15,127 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:15,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-09 01:55:15,528 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:15,528 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing abbc0d228ef2735615e5e8f7e1c367fc, disabling compactions & flushes 2023-03-09 01:55:15,528 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:15,528 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:15,528 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. after waiting 0 ms 2023-03-09 01:55:15,528 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:15,528 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:15,528 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:15,531 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:55:15,531 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326915531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326915531"}]},"ts":"1678326915531"} 2023-03-09 01:55:15,532 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326915531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326915531"}]},"ts":"1678326915531"} 2023-03-09 01:55:15,532 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326915531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326915531"}]},"ts":"1678326915531"} 2023-03-09 01:55:15,533 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-09 01:55:15,534 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:55:15,534 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326915534"}]},"ts":"1678326915534"} 2023-03-09 01:55:15,535 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-09 01:55:15,540 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, ASSIGN}] 2023-03-09 01:55:15,541 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, ASSIGN 2023-03-09 01:55:15,541 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, ASSIGN 2023-03-09 01:55:15,541 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, ASSIGN 2023-03-09 01:55:15,542 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:55:15,542 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:55:15,542 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:55:15,693 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=abbc0d228ef2735615e5e8f7e1c367fc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:15,693 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=18e9a9ff9da4b5322903ba24ba13b2e1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:15,694 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326915693"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326915693"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326915693"}]},"ts":"1678326915693"} 2023-03-09 01:55:15,693 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=00f768e6a69add6056570cdcb3ad746e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:15,694 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326915693"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326915693"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326915693"}]},"ts":"1678326915693"} 2023-03-09 01:55:15,694 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326915693"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326915693"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326915693"}]},"ts":"1678326915693"} 2023-03-09 01:55:15,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure abbc0d228ef2735615e5e8f7e1c367fc, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:15,696 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 18e9a9ff9da4b5322903ba24ba13b2e1, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:15,698 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure 00f768e6a69add6056570cdcb3ad746e, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:15,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-09 01:55:15,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:15,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 18e9a9ff9da4b5322903ba24ba13b2e1, NAME => 'testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.', STARTKEY => '1', ENDKEY => '2'} 2023-03-09 01:55:15,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:15,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:15,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:15,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:15,853 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:15,854 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1 2023-03-09 01:55:15,854 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1 2023-03-09 01:55:15,854 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18e9a9ff9da4b5322903ba24ba13b2e1 columnFamilyName family1 2023-03-09 01:55:15,855 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(310): Store=18e9a9ff9da4b5322903ba24ba13b2e1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:15,855 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:15,856 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2 2023-03-09 01:55:15,856 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2 2023-03-09 01:55:15,857 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18e9a9ff9da4b5322903ba24ba13b2e1 columnFamilyName family2 2023-03-09 01:55:15,858 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(310): Store=18e9a9ff9da4b5322903ba24ba13b2e1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:15,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:15,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:15,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:15,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:15,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:55:15,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 18e9a9ff9da4b5322903ba24ba13b2e1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66791930, jitterRate=-0.004722684621810913}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:15,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:15,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1., pid=84, masterSystemTime=1678326915848 2023-03-09 01:55:15,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:15,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:15,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:15,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => abbc0d228ef2735615e5e8f7e1c367fc, NAME => 'testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.', STARTKEY => '2', ENDKEY => ''} 2023-03-09 01:55:15,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:15,866 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=18e9a9ff9da4b5322903ba24ba13b2e1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:15,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:15,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:15,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:15,866 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326915866"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326915866"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326915866"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326915866"}]},"ts":"1678326915866"} 2023-03-09 01:55:15,867 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:15,868 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1 2023-03-09 01:55:15,868 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1 2023-03-09 01:55:15,868 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-03-09 01:55:15,868 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 18e9a9ff9da4b5322903ba24ba13b2e1, server=jenkins-hbase4.apache.org,46511,1678326819757 in 171 msec 2023-03-09 01:55:15,868 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region abbc0d228ef2735615e5e8f7e1c367fc columnFamilyName family1 2023-03-09 01:55:15,869 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(310): Store=abbc0d228ef2735615e5e8f7e1c367fc/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:15,869 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, ASSIGN in 330 msec 2023-03-09 01:55:15,870 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:15,871 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2 2023-03-09 01:55:15,871 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2 2023-03-09 01:55:15,871 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region abbc0d228ef2735615e5e8f7e1c367fc columnFamilyName family2 2023-03-09 01:55:15,872 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(310): Store=abbc0d228ef2735615e5e8f7e1c367fc/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:15,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:15,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:15,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:15,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:15,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:55:15,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened abbc0d228ef2735615e5e8f7e1c367fc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67759061, jitterRate=0.009688690304756165}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:15,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:15,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc., pid=83, masterSystemTime=1678326915848 2023-03-09 01:55:15,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:15,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:15,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:15,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 00f768e6a69add6056570cdcb3ad746e, NAME => 'testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.', STARTKEY => '', ENDKEY => '1'} 2023-03-09 01:55:15,879 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=abbc0d228ef2735615e5e8f7e1c367fc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:15,880 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326915879"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326915879"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326915879"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326915879"}]},"ts":"1678326915879"} 2023-03-09 01:55:15,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:15,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:15,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:15,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:15,881 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:15,882 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-03-09 01:55:15,882 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure abbc0d228ef2735615e5e8f7e1c367fc, server=jenkins-hbase4.apache.org,46511,1678326819757 in 186 msec 2023-03-09 01:55:15,883 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1 2023-03-09 01:55:15,883 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1 2023-03-09 01:55:15,883 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 00f768e6a69add6056570cdcb3ad746e columnFamilyName family1 2023-03-09 01:55:15,883 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, ASSIGN in 342 msec 2023-03-09 01:55:15,884 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(310): Store=00f768e6a69add6056570cdcb3ad746e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:15,884 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:15,885 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2 2023-03-09 01:55:15,885 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2 2023-03-09 01:55:15,886 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 00f768e6a69add6056570cdcb3ad746e columnFamilyName family2 2023-03-09 01:55:15,886 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(310): Store=00f768e6a69add6056570cdcb3ad746e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:15,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:15,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:15,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:15,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:15,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:55:15,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 00f768e6a69add6056570cdcb3ad746e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65936221, jitterRate=-0.017473742365837097}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:15,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:15,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e., pid=85, masterSystemTime=1678326915848 2023-03-09 01:55:15,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:15,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:15,894 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=00f768e6a69add6056570cdcb3ad746e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:15,894 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326915894"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326915894"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326915894"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326915894"}]},"ts":"1678326915894"} 2023-03-09 01:55:15,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-03-09 01:55:15,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure 00f768e6a69add6056570cdcb3ad746e, server=jenkins-hbase4.apache.org,46511,1678326819757 in 199 msec 2023-03-09 01:55:15,898 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-03-09 01:55:15,898 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, ASSIGN in 359 msec 2023-03-09 01:55:15,899 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:55:15,899 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326915899"}]},"ts":"1678326915899"} 2023-03-09 01:55:15,900 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-09 01:55:15,903 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:55:15,904 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2100 sec 2023-03-09 01:55:16,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-09 01:55:16,801 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-09 01:55:16,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:16,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:16,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:16,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:16,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-09 01:55:16,815 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-09 01:55:16,815 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:16,815 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-09 01:55:16,815 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-09 01:55:16,815 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:16,815 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:16,816 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:16,816 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:16,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:16,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:16,817 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:16,817 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:16,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:16,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:16,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:16,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:16,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-09 01:55:16,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-09 01:55:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-09 01:55:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-09 01:55:16,818 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:16,818 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:16,818 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:16,818 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:16,818 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. started... 2023-03-09 01:55:16,818 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. started... 2023-03-09 01:55:16,818 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. started... 2023-03-09 01:55:16,819 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 18e9a9ff9da4b5322903ba24ba13b2e1 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:55:16,819 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 00f768e6a69add6056570cdcb3ad746e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:55:16,820 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing abbc0d228ef2735615e5e8f7e1c367fc 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:55:17,241 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914 2023-03-09 01:55:17,241 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7 2023-03-09 01:55:17,242 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96 2023-03-09 01:55:17,559 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-09 01:55:17,651 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/bd7eaca3d0bb49c881093d8bffbe4a61 2023-03-09 01:55:17,654 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/6b9df6dda32a470bb7beedc493b4d7c2 2023-03-09 01:55:17,654 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/554613f5dee54006ad0ca4cc98415b67 2023-03-09 01:55:18,070 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309e0f7092228534ccabbe807808edc1f89 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309e0f7092228534ccabbe807808edc1f89 2023-03-09 01:55:18,070 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303090f8e236dd2b44d0aa74f234ad0abaf0b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303090f8e236dd2b44d0aa74f234ad0abaf0b 2023-03-09 01:55:18,070 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309fec823224a2f4e7abc862491640daf44 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309fec823224a2f4e7abc862491640daf44 2023-03-09 01:55:18,476 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/51a937efd2f644babe8351499c9fa15a 2023-03-09 01:55:18,476 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/def1134fc58a4a269289ed9ac51b532e 2023-03-09 01:55:18,476 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/60bbe8792b3d43189452c94ff064687b 2023-03-09 01:55:18,482 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/bd7eaca3d0bb49c881093d8bffbe4a61 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/bd7eaca3d0bb49c881093d8bffbe4a61 2023-03-09 01:55:18,482 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/6b9df6dda32a470bb7beedc493b4d7c2 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/6b9df6dda32a470bb7beedc493b4d7c2 2023-03-09 01:55:18,482 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/554613f5dee54006ad0ca4cc98415b67 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/554613f5dee54006ad0ca4cc98415b67 2023-03-09 01:55:18,486 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/bd7eaca3d0bb49c881093d8bffbe4a61, entries=4, sequenceid=5, filesize=5.4 K 2023-03-09 01:55:18,487 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/554613f5dee54006ad0ca4cc98415b67, entries=4, sequenceid=5, filesize=5.4 K 2023-03-09 01:55:18,487 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/6b9df6dda32a470bb7beedc493b4d7c2, entries=4, sequenceid=5, filesize=5.4 K 2023-03-09 01:55:18,487 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/51a937efd2f644babe8351499c9fa15a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/51a937efd2f644babe8351499c9fa15a 2023-03-09 01:55:18,487 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/def1134fc58a4a269289ed9ac51b532e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/def1134fc58a4a269289ed9ac51b532e 2023-03-09 01:55:18,488 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/60bbe8792b3d43189452c94ff064687b as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/60bbe8792b3d43189452c94ff064687b 2023-03-09 01:55:18,493 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/51a937efd2f644babe8351499c9fa15a, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:55:18,494 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/def1134fc58a4a269289ed9ac51b532e, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:55:18,494 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 00f768e6a69add6056570cdcb3ad746e in 1675ms, sequenceid=5, compaction requested=false 2023-03-09 01:55:18,494 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/60bbe8792b3d43189452c94ff064687b, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:55:18,495 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:18,495 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:18,495 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for abbc0d228ef2735615e5e8f7e1c367fc in 1675ms, sequenceid=5, compaction requested=false 2023-03-09 01:55:18,495 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:18,495 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:18,495 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:18,495 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:18,495 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 18e9a9ff9da4b5322903ba24ba13b2e1 in 1676ms, sequenceid=5, compaction requested=false 2023-03-09 01:55:18,496 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:18,496 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:18,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:18,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:18,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-09 01:55:18,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-09 01:55:18,497 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:18,497 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:18,498 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:18,498 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:18,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-09 01:55:18,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18dce162[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:18,499 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-09 01:55:18,499 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:18,501 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:18,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:18,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:18,501 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-09 01:55:18,501 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-09 01:55:18,501 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,501 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:18,501 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:18,502 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-09 01:55:18,502 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,502 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:18,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:18,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:18,502 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-09 01:55:18,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:18,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:18,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:18,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:18,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:18,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:18,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-09 01:55:18,505 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-09 01:55:18,505 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:18,505 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:18,505 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-09 01:55:18,505 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:18,508 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,508 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:18,508 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,508 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:18,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:18,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:18,509 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:18,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:18,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:18,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,511 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,511 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:18,511 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:18,511 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:18,511 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-09 01:55:18,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:18,511 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:18,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:18,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:18,511 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:18,511 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:18,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-09 01:55:18,512 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:18,512 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:18,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:18,512 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:18,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:18,612 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:18,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:18,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:18,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:18,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:18,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:18,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:18,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-09 01:55:18,619 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-09 01:55:18,619 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:18,619 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-09 01:55:18,619 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-09 01:55:18,619 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,620 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:18,621 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,621 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:18,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:18,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:18,621 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:18,621 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:18,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:18,622 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:18,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:18,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:18,622 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-09 01:55:18,623 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-09 01:55:18,623 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-09 01:55:18,623 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-09 01:55:18,623 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:18,623 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-09 01:55:18,623 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:18,623 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:18,623 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:18,623 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:18,623 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:18,623 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. started... 2023-03-09 01:55:18,623 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. started... 2023-03-09 01:55:18,623 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. started... 2023-03-09 01:55:18,623 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 00f768e6a69add6056570cdcb3ad746e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:55:18,623 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 18e9a9ff9da4b5322903ba24ba13b2e1 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:55:18,623 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing abbc0d228ef2735615e5e8f7e1c367fc 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:55:18,650 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178 2023-03-09 01:55:18,650 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99 2023-03-09 01:55:19,041 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412 2023-03-09 01:55:19,059 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/46f048b3bec44725bbd2a2dd1547ad65 2023-03-09 01:55:19,060 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/a19058eff3f84d83a01b0191d4a9958d 2023-03-09 01:55:19,447 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/9f28ae193d6b4e34bbb73a959c56c0f2 2023-03-09 01:55:19,476 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309aa9b7d5917264d269dc6eeff4e21a1c5 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309aa9b7d5917264d269dc6eeff4e21a1c5 2023-03-09 01:55:19,477 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030927a1a49fb1f04db4bb45b3beaecf8dc6 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030927a1a49fb1f04db4bb45b3beaecf8dc6 2023-03-09 01:55:19,862 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303093df924d792694c629ae9256c3bc125ef to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303093df924d792694c629ae9256c3bc125ef 2023-03-09 01:55:19,882 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/647e37787cd149a190171dfdcdb08187 2023-03-09 01:55:19,886 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/46f048b3bec44725bbd2a2dd1547ad65 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/46f048b3bec44725bbd2a2dd1547ad65 2023-03-09 01:55:19,889 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/46f048b3bec44725bbd2a2dd1547ad65, entries=4, sequenceid=9, filesize=5.4 K 2023-03-09 01:55:19,889 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/e23c8b4c586a48068d85ad2125091e46 2023-03-09 01:55:19,890 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/647e37787cd149a190171dfdcdb08187 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/647e37787cd149a190171dfdcdb08187 2023-03-09 01:55:19,893 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/a19058eff3f84d83a01b0191d4a9958d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a19058eff3f84d83a01b0191d4a9958d 2023-03-09 01:55:19,893 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/647e37787cd149a190171dfdcdb08187, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:55:19,894 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 18e9a9ff9da4b5322903ba24ba13b2e1 in 1271ms, sequenceid=9, compaction requested=false 2023-03-09 01:55:19,894 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:19,894 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:19,894 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:19,897 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a19058eff3f84d83a01b0191d4a9958d, entries=4, sequenceid=9, filesize=5.4 K 2023-03-09 01:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/e23c8b4c586a48068d85ad2125091e46 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/e23c8b4c586a48068d85ad2125091e46 2023-03-09 01:55:19,902 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/e23c8b4c586a48068d85ad2125091e46, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:55:19,902 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for abbc0d228ef2735615e5e8f7e1c367fc in 1279ms, sequenceid=9, compaction requested=false 2023-03-09 01:55:19,902 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:19,902 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:19,903 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:19,918 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:55:20,267 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/b54f40cf492f48f2a959cbed0a4fd66d 2023-03-09 01:55:20,272 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/9f28ae193d6b4e34bbb73a959c56c0f2 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/9f28ae193d6b4e34bbb73a959c56c0f2 2023-03-09 01:55:20,276 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/9f28ae193d6b4e34bbb73a959c56c0f2, entries=4, sequenceid=9, filesize=5.4 K 2023-03-09 01:55:20,277 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/b54f40cf492f48f2a959cbed0a4fd66d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/b54f40cf492f48f2a959cbed0a4fd66d 2023-03-09 01:55:20,282 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/b54f40cf492f48f2a959cbed0a4fd66d, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:55:20,282 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 00f768e6a69add6056570cdcb3ad746e in 1659ms, sequenceid=9, compaction requested=false 2023-03-09 01:55:20,283 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:20,283 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:20,283 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:20,283 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:20,283 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,283 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-09 01:55:20,283 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-09 01:55:20,284 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:20,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:20,285 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:20,285 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:20,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:20,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-09 01:55:20,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5398def8[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:20,286 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-09 01:55:20,286 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:20,289 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:20,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:20,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:20,289 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-09 01:55:20,289 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-09 01:55:20,289 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:20,289 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,289 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:20,290 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-09 01:55:20,290 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:20,290 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,290 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-09 01:55:20,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:20,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:20,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:20,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:20,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:20,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:20,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:20,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:20,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-09 01:55:20,292 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-09 01:55:20,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:20,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:20,292 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-09 01:55:20,293 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:20,294 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,294 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:20,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:20,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,294 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:20,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,295 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:20,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:20,296 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:20,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:20,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:20,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:20,298 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:20,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-09 01:55:20,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:20,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,298 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:20,299 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:20,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:20,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:20,399 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:20,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:20,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:20,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:20,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:20,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:20,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:20,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-09 01:55:20,406 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-09 01:55:20,406 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:20,406 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-09 01:55:20,406 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-09 01:55:20,406 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,406 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:20,409 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,409 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:20,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:20,409 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:20,409 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:20,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:20,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:20,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:20,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-09 01:55:20,410 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-09 01:55:20,410 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-09 01:55:20,410 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-09 01:55:20,410 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:20,410 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-09 01:55:20,410 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:20,410 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. started... 2023-03-09 01:55:20,410 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:20,411 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:20,411 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:20,411 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. started... 2023-03-09 01:55:20,411 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. started... 2023-03-09 01:55:20,411 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing abbc0d228ef2735615e5e8f7e1c367fc 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:55:20,411 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 18e9a9ff9da4b5322903ba24ba13b2e1 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:55:20,411 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:20,411 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 00f768e6a69add6056570cdcb3ad746e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:55:20,433 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5 2023-03-09 01:55:20,829 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa 2023-03-09 01:55:20,831 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84 2023-03-09 01:55:20,836 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/c03f67bd5eda426d98fe2d2f23cddf3c 2023-03-09 01:55:20,838 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/e52084fb3ce043d3b3193b82c7c8c28d 2023-03-09 01:55:20,839 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/82dcda8270744c87b9890f8a8573eb1a 2023-03-09 01:55:21,255 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309bb4349f7ac904ce084506bf4674e0c82 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309bb4349f7ac904ce084506bf4674e0c82 2023-03-09 01:55:21,256 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309b7ed83e86ebb480fbb10faee5ca824c1 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309b7ed83e86ebb480fbb10faee5ca824c1 2023-03-09 01:55:21,256 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309b28a61ad448d408b804acac511dc7db0 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309b28a61ad448d408b804acac511dc7db0 2023-03-09 01:55:21,662 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/f626b340147b41f481b99eceb82737c3 2023-03-09 01:55:21,662 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/8c204b7188634b12a4b18c8cb9e96815 2023-03-09 01:55:21,665 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/e8e622fe17cc4feaaa821052b05b2f1d 2023-03-09 01:55:21,668 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/82dcda8270744c87b9890f8a8573eb1a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/82dcda8270744c87b9890f8a8573eb1a 2023-03-09 01:55:21,668 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/c03f67bd5eda426d98fe2d2f23cddf3c as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/c03f67bd5eda426d98fe2d2f23cddf3c 2023-03-09 01:55:21,670 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/e52084fb3ce043d3b3193b82c7c8c28d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/e52084fb3ce043d3b3193b82c7c8c28d 2023-03-09 01:55:21,672 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/82dcda8270744c87b9890f8a8573eb1a, entries=4, sequenceid=13, filesize=5.4 K 2023-03-09 01:55:21,672 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/c03f67bd5eda426d98fe2d2f23cddf3c, entries=4, sequenceid=13, filesize=5.4 K 2023-03-09 01:55:21,673 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/8c204b7188634b12a4b18c8cb9e96815 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/8c204b7188634b12a4b18c8cb9e96815 2023-03-09 01:55:21,673 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/f626b340147b41f481b99eceb82737c3 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/f626b340147b41f481b99eceb82737c3 2023-03-09 01:55:21,674 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/e52084fb3ce043d3b3193b82c7c8c28d, entries=4, sequenceid=13, filesize=5.4 K 2023-03-09 01:55:21,674 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/e8e622fe17cc4feaaa821052b05b2f1d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/e8e622fe17cc4feaaa821052b05b2f1d 2023-03-09 01:55:21,677 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/f626b340147b41f481b99eceb82737c3, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:55:21,678 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for abbc0d228ef2735615e5e8f7e1c367fc in 1267ms, sequenceid=13, compaction requested=true 2023-03-09 01:55:21,678 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:21,678 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:21,678 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:21,678 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/8c204b7188634b12a4b18c8cb9e96815, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:55:21,679 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/e8e622fe17cc4feaaa821052b05b2f1d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:55:21,679 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 00f768e6a69add6056570cdcb3ad746e in 1268ms, sequenceid=13, compaction requested=true 2023-03-09 01:55:21,679 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:21,679 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:21,679 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:21,679 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 18e9a9ff9da4b5322903ba24ba13b2e1 in 1268ms, sequenceid=13, compaction requested=true 2023-03-09 01:55:21,679 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:21,679 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:21,679 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:21,679 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:21,680 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,680 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-09 01:55:21,680 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-09 01:55:21,681 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,681 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:21,682 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:21,682 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:21,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-09 01:55:21,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6909ff7e[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:21,683 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-09 01:55:21,683 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:21,684 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:21,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:21,684 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,685 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-09 01:55:21,685 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-09 01:55:21,685 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:21,685 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:21,686 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,686 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-09 01:55:21,686 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:21,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:21,686 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-09 01:55:21,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:21,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:21,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:21,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-09 01:55:21,688 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-09 01:55:21,689 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:21,689 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:21,689 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-09 01:55:21,689 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:21,694 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,694 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:21,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:21,694 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:21,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:21,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:21,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:21,695 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:21,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:21,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:21,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,696 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:21,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:21,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:21,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:21,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:21,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:21,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:21,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-09 01:55:21,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:21,698 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:21,698 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,699 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:21,699 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:21,699 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,699 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:21,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:21,699 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:21,799 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:21,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:21,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:21,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:21,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:21,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:21,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:21,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-09 01:55:21,808 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-09 01:55:21,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:21,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-09 01:55:21,808 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-09 01:55:21,809 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,809 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:21,810 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,810 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:21,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:21,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:21,810 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:21,810 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:21,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:21,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:21,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:21,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:21,811 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-09 01:55:21,813 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-09 01:55:21,813 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-09 01:55:21,813 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-09 01:55:21,813 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:21,813 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-09 01:55:21,814 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:21,814 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. started... 2023-03-09 01:55:21,814 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:21,814 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. started... 2023-03-09 01:55:21,814 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing abbc0d228ef2735615e5e8f7e1c367fc 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:55:21,814 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 18e9a9ff9da4b5322903ba24ba13b2e1 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:55:21,814 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:21,814 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:21,814 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:21,814 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. started... 2023-03-09 01:55:21,815 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 00f768e6a69add6056570cdcb3ad746e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:55:21,854 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763 2023-03-09 01:55:22,245 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b 2023-03-09 01:55:22,245 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f 2023-03-09 01:55:22,254 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/c092b1b2898b430cb5a6fd6ef235b622 2023-03-09 01:55:22,294 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:55:22,294 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x492fa0d3 to 127.0.0.1:50548 2023-03-09 01:55:22,294 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/9a43c7ed86e441a396bd23845a381994 2023-03-09 01:55:22,294 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:55:22,294 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:55:22,294 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5263d421 to 127.0.0.1:50548 2023-03-09 01:55:22,294 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:55:22,295 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1597f568 to 127.0.0.1:50548 2023-03-09 01:55:22,295 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:55:22,295 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:55:22,295 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3cff5645 to 127.0.0.1:50548 2023-03-09 01:55:22,295 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:55:22,295 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:55:22,295 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e27fd0a to 127.0.0.1:50548 2023-03-09 01:55:22,295 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:55:22,295 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x220bc4f3 to 127.0.0.1:50548 2023-03-09 01:55:22,295 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:55:22,315 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309ee66da52298848a4a316e031c7adb822 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309ee66da52298848a4a316e031c7adb822 2023-03-09 01:55:22,655 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/c8a9da0c9dea4201985e0dda7f99ce78 2023-03-09 01:55:22,701 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309beb0560b7a674037913ac15ba3bb037f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309beb0560b7a674037913ac15ba3bb037f 2023-03-09 01:55:22,720 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/5c406707c2a64cd4ab677f3124d6af50 2023-03-09 01:55:22,724 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/9a43c7ed86e441a396bd23845a381994 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/9a43c7ed86e441a396bd23845a381994 2023-03-09 01:55:22,727 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/9a43c7ed86e441a396bd23845a381994, entries=4, sequenceid=17, filesize=5.4 K 2023-03-09 01:55:22,728 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/5c406707c2a64cd4ab677f3124d6af50 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/5c406707c2a64cd4ab677f3124d6af50 2023-03-09 01:55:22,731 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/5c406707c2a64cd4ab677f3124d6af50, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:55:22,732 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for abbc0d228ef2735615e5e8f7e1c367fc in 918ms, sequenceid=17, compaction requested=true 2023-03-09 01:55:22,732 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:22,732 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:22,732 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:23,070 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309251d64456774424d9d2445a93d454feb to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309251d64456774424d9d2445a93d454feb 2023-03-09 01:55:23,106 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/0a6c637662964e48a5d1d38c97d82cef 2023-03-09 01:55:23,110 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/c092b1b2898b430cb5a6fd6ef235b622 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/c092b1b2898b430cb5a6fd6ef235b622 2023-03-09 01:55:23,113 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/c092b1b2898b430cb5a6fd6ef235b622, entries=4, sequenceid=17, filesize=5.4 K 2023-03-09 01:55:23,114 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/0a6c637662964e48a5d1d38c97d82cef as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/0a6c637662964e48a5d1d38c97d82cef 2023-03-09 01:55:23,117 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/0a6c637662964e48a5d1d38c97d82cef, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:55:23,118 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 18e9a9ff9da4b5322903ba24ba13b2e1 in 1304ms, sequenceid=17, compaction requested=true 2023-03-09 01:55:23,118 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:23,118 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:23,118 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:23,475 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/6fd2b3985d2b4c7b812d559361310f38 2023-03-09 01:55:23,479 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/c8a9da0c9dea4201985e0dda7f99ce78 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/c8a9da0c9dea4201985e0dda7f99ce78 2023-03-09 01:55:23,483 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/c8a9da0c9dea4201985e0dda7f99ce78, entries=4, sequenceid=17, filesize=5.4 K 2023-03-09 01:55:23,483 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/6fd2b3985d2b4c7b812d559361310f38 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/6fd2b3985d2b4c7b812d559361310f38 2023-03-09 01:55:23,487 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/6fd2b3985d2b4c7b812d559361310f38, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:55:23,488 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 00f768e6a69add6056570cdcb3ad746e in 1673ms, sequenceid=17, compaction requested=true 2023-03-09 01:55:23,488 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:23,488 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:23,489 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:23,489 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:23,489 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,489 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-09 01:55:23,489 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-09 01:55:23,490 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,490 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:23,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:23,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:23,491 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:23,491 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:23,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:23,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:23,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:23,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:23,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-09 01:55:23,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77c5f399[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:23,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-09 01:55:23,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:23,499 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:23,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:23,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:23,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-09 01:55:23,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-09 01:55:23,499 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:23,499 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,500 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:23,501 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-09 01:55:23,501 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:23,501 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,501 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-09 01:55:23,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:23,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:23,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:23,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:23,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:23,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:23,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:23,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:23,503 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-09 01:55:23,503 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:23,503 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:23,503 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-09 01:55:23,503 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:23,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-09 01:55:23,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:23,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:23,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:23,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:23,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:23,505 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:23,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:23,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:23,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:23,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:23,506 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:23,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:23,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:23,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:23,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:23,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,507 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:23,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:23,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:23,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:23,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:23,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:23,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:23,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-09 01:55:23,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:23,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:23,509 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:23,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-09 01:55:23,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:23,509 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:23,510 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:23,510 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:23,510 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:23,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:23,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:23,510 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:23,610 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:23,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:23,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-09 01:55:23,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678326923621 (current time:1678326923621). 2023-03-09 01:55:23,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-09 01:55:23,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] zookeeper.ReadOnlyZKClient(139): Connect 0x4f038af7 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:23,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e23509d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:23,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:55:23,637 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39912, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:55:23,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4f038af7 to 127.0.0.1:50548 2023-03-09 01:55:23,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:55:23,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-09 01:55:23,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-09 01:55:23,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-09 01:55:23,651 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-09 01:55:23,652 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678327523651 2023-03-09 01:55:23,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-09 01:55:23,652 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1678326923611 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-09 01:55:23,653 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-09 01:55:23,654 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-09 01:55:23,654 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 4 msec 2023-03-09 01:55:23,655 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-09 01:55:23,655 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1678327523655 2023-03-09 01:55:23,658 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:23,658 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-09 01:55:23,758 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:23,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:23,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-09 01:55:23,763 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-09 01:55:23,963 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:23,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:23,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-09 01:55:23,964 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-09 01:55:24,062 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1678326923611 2023-03-09 01:55:24,062 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1678326923611' 2023-03-09 01:55:24,062 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-09 01:55:24,062 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1678326923611' starting 'acquire' 2023-03-09 01:55:24,062 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1678326923611', kicking off acquire phase on members. 2023-03-09 01:55:24,063 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,063 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1678326923611 2023-03-09 01:55:24,064 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,064 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-09 01:55:24,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-09 01:55:24,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-09 01:55:24,064 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1678326923611 2023-03-09 01:55:24,064 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:24,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-09 01:55:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1678326923611 2023-03-09 01:55:24,065 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1678326923611 from table testCompactionWithHFileLink type FLUSH 2023-03-09 01:55:24,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1678326923611 2023-03-09 01:55:24,067 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1678326923611' with timeout 300000ms 2023-03-09 01:55:24,068 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-09 01:55:24,068 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1678326923611' starting 'acquire' stage 2023-03-09 01:55:24,068 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1678326923611' locally acquired 2023-03-09 01:55:24,068 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (snaptb-1678326923611) in zk 2023-03-09 01:55:24,069 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,069 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1678326923611 2023-03-09 01:55:24,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:24,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-09 01:55:24,069 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678326923611 2023-03-09 01:55:24,069 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1678326923611' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:24,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:24,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:24,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678326923611 2023-03-09 01:55:24,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:24,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'snaptb-1678326923611' on coordinator 2023-03-09 01:55:24,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@86ddbf2[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:24,071 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1678326923611' starting 'in-barrier' execution. 2023-03-09 01:55:24,071 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1678326923611 2023-03-09 01:55:24,075 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678326923611 2023-03-09 01:55:24,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1678326923611 2023-03-09 01:55:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1678326923611 2023-03-09 01:55:24,076 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1678326923611' received 'reached' from coordinator. 2023-03-09 01:55:24,076 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,076 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:24,077 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:24,077 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. started... 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. started... 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. started... 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:24,077 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:24,078 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.' region-info for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,078 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.' region-info for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,078 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.' region-info for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,083 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-09 01:55:24,083 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-09 01:55:24,083 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-09 01:55:24,087 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/bd7eaca3d0bb49c881093d8bffbe4a61, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/9f28ae193d6b4e34bbb73a959c56c0f2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/82dcda8270744c87b9890f8a8573eb1a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/c8a9da0c9dea4201985e0dda7f99ce78] hfiles 2023-03-09 01:55:24,087 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/6b9df6dda32a470bb7beedc493b4d7c2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/46f048b3bec44725bbd2a2dd1547ad65, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/e52084fb3ce043d3b3193b82c7c8c28d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/c092b1b2898b430cb5a6fd6ef235b622] hfiles 2023-03-09 01:55:24,087 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/bd7eaca3d0bb49c881093d8bffbe4a61 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,087 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/6b9df6dda32a470bb7beedc493b4d7c2 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,087 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/554613f5dee54006ad0ca4cc98415b67, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a19058eff3f84d83a01b0191d4a9958d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/c03f67bd5eda426d98fe2d2f23cddf3c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/9a43c7ed86e441a396bd23845a381994] hfiles 2023-03-09 01:55:24,087 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/554613f5dee54006ad0ca4cc98415b67 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,248 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-09 01:55:24,250 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/46f048b3bec44725bbd2a2dd1547ad65 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,250 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a19058eff3f84d83a01b0191d4a9958d for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,250 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/9f28ae193d6b4e34bbb73a959c56c0f2 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/c03f67bd5eda426d98fe2d2f23cddf3c for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,250 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/e52084fb3ce043d3b3193b82c7c8c28d for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/9a43c7ed86e441a396bd23845a381994 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/c092b1b2898b430cb5a6fd6ef235b622 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/82dcda8270744c87b9890f8a8573eb1a for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/c8a9da0c9dea4201985e0dda7f99ce78 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/51a937efd2f644babe8351499c9fa15a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/b54f40cf492f48f2a959cbed0a4fd66d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/8c204b7188634b12a4b18c8cb9e96815, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/6fd2b3985d2b4c7b812d559361310f38] hfiles 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/51a937efd2f644babe8351499c9fa15a for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/def1134fc58a4a269289ed9ac51b532e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/e23c8b4c586a48068d85ad2125091e46, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/f626b340147b41f481b99eceb82737c3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/5c406707c2a64cd4ab677f3124d6af50] hfiles 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/def1134fc58a4a269289ed9ac51b532e for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/60bbe8792b3d43189452c94ff064687b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/647e37787cd149a190171dfdcdb08187, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/e8e622fe17cc4feaaa821052b05b2f1d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/0a6c637662964e48a5d1d38c97d82cef] hfiles 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/60bbe8792b3d43189452c94ff064687b for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/e23c8b4c586a48068d85ad2125091e46 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/b54f40cf492f48f2a959cbed0a4fd66d for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/8c204b7188634b12a4b18c8cb9e96815 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/6fd2b3985d2b4c7b812d559361310f38 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/f626b340147b41f481b99eceb82737c3 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/647e37787cd149a190171dfdcdb08187 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/5c406707c2a64cd4ab677f3124d6af50 for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/e8e622fe17cc4feaaa821052b05b2f1d for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,251 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/0a6c637662964e48a5d1d38c97d82cef for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,265 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:24,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:24,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-09 01:55:24,266 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-09 01:55:24,662 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. completed. 2023-03-09 01:55:24,662 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:24,662 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. completed. 2023-03-09 01:55:24,663 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:24,663 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-09 01:55:24,663 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-09 01:55:24,663 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. completed. 2023-03-09 01:55:24,663 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:24,663 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-09 01:55:24,663 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-09 01:55:24,663 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,663 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1678326923611' locally completed 2023-03-09 01:55:24,663 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1678326923611' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:24,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1678326923611' has notified controller of completion 2023-03-09 01:55:24,665 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:24,665 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1678326923611' completed. 2023-03-09 01:55:24,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:24,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-09 01:55:24,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:24,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:24,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678326923611 2023-03-09 01:55:24,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:24,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678326923611 2023-03-09 01:55:24,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1678326923611' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:24,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'snaptb-1678326923611', counting down latch. Waiting for 0 more 2023-03-09 01:55:24,667 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1678326923611' execution completed 2023-03-09 01:55:24,667 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:24,667 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:24,667 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1678326923611 2023-03-09 01:55:24,667 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1678326923611including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-09 01:55:24,669 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,669 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:24,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-09 01:55:24,669 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-09 01:55:24,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,670 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-09 01:55:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-09 01:55:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678326923611 2023-03-09 01:55:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:24,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678326923611 2023-03-09 01:55:24,671 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1678326923611 2023-03-09 01:55:24,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-09 01:55:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-09 01:55:24,673 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1678326923611 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-09 01:55:24,673 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:24,673 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678326923611 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1678326923611 2023-03-09 01:55:24,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-09 01:55:24,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-09 01:55:24,673 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1678326923611 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678326923611/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:24,673 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678326923611 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1678326923611 2023-03-09 01:55:24,673 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1678326923611 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412] mob files 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914 2023-03-09 01:55:24,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030927a1a49fb1f04db4bb45b3beaecf8dc6, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309b7ed83e86ebb480fbb10faee5ca824c1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309beb0560b7a674037913ac15ba3bb037f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309fec823224a2f4e7abc862491640daf44, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309aa9b7d5917264d269dc6eeff4e21a1c5, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309bb4349f7ac904ce084506bf4674e0c82, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309e0f7092228534ccabbe807808edc1f89, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309ee66da52298848a4a316e031c7adb822, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303090f8e236dd2b44d0aa74f234ad0abaf0b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309251d64456774424d9d2445a93d454feb, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303093df924d792694c629ae9256c3bc125ef, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309b28a61ad448d408b804acac511dc7db0] mob files 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030927a1a49fb1f04db4bb45b3beaecf8dc6 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309b7ed83e86ebb480fbb10faee5ca824c1 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309beb0560b7a674037913ac15ba3bb037f 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309fec823224a2f4e7abc862491640daf44 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309aa9b7d5917264d269dc6eeff4e21a1c5 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309bb4349f7ac904ce084506bf4674e0c82 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309e0f7092228534ccabbe807808edc1f89 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309ee66da52298848a4a316e031c7adb822 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303090f8e236dd2b44d0aa74f234ad0abaf0b 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309251d64456774424d9d2445a93d454feb 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303093df924d792694c629ae9256c3bc125ef 2023-03-09 01:55:24,677 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309b28a61ad448d408b804acac511dc7db0 2023-03-09 01:55:24,766 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:24,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:24,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-09 01:55:24,767 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-09 01:55:25,083 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1678326923611 2023-03-09 01:55:25,085 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp/snaptb-1678326923611 2023-03-09 01:55:25,546 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp/snaptb-1678326923611 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/snaptb-1678326923611 2023-03-09 01:55:25,569 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1678326923611 of table testCompactionWithHFileLink completed 2023-03-09 01:55:25,570 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp/snaptb-1678326923611 2023-03-09 01:55:25,570 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp/snaptb-1678326923611 2023-03-09 01:55:25,570 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1678326923611 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678326923652Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1678326925083 (+1431 ms)Consolidate snapshot: snaptb-1678326923611 at 1678326925083Loading Region manifests for snaptb-1678326923611 at 1678326925083Writing data manifest for snaptb-1678326923611 at 1678326925098 (+15 ms)Verifying snapshot: snaptb-1678326923611 at 1678326925508 (+410 ms)Snapshot snaptb-1678326923611 of table testCompactionWithHFileLink completed at 1678326925569 (+61 ms) 2023-03-09 01:55:25,571 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-09 01:55:25,573 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.9190 sec 2023-03-09 01:55:25,767 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:25,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:25,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-09 01:55:25,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:25,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:25,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-09 01:55:25,800 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-09 01:55:25,800 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:25,800 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-09 01:55:25,800 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-09 01:55:25,800 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:25,800 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:25,802 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:25,802 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:25,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:25,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:25,803 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:25,803 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:25,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:25,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:25,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:25,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:25,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-09 01:55:25,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-09 01:55:25,804 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-09 01:55:25,804 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:25,804 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-09 01:55:25,804 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:25,804 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. started... 2023-03-09 01:55:25,804 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:25,804 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. started... 2023-03-09 01:55:25,804 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing abbc0d228ef2735615e5e8f7e1c367fc 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:55:25,804 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 18e9a9ff9da4b5322903ba24ba13b2e1 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:55:25,806 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:25,806 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:25,806 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:25,806 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. started... 2023-03-09 01:55:25,806 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 00f768e6a69add6056570cdcb3ad746e 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:55:26,218 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/72f56cdcf0b942a1ba079807eeaf795a 2023-03-09 01:55:26,221 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/68c9f8cd8fa04baeaffaa91df581cd38 2023-03-09 01:55:26,223 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72f56cdcf0b942a1ba079807eeaf795a 2023-03-09 01:55:26,224 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68c9f8cd8fa04baeaffaa91df581cd38 2023-03-09 01:55:26,619 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/f7ffccdc37e24c36926633d1d4c0a3dc 2023-03-09 01:55:26,623 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7ffccdc37e24c36926633d1d4c0a3dc 2023-03-09 01:55:26,641 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/00c7e796853b4a5e9b3384925b442817 2023-03-09 01:55:26,645 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00c7e796853b4a5e9b3384925b442817 2023-03-09 01:55:26,646 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/72f56cdcf0b942a1ba079807eeaf795a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/72f56cdcf0b942a1ba079807eeaf795a 2023-03-09 01:55:26,650 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72f56cdcf0b942a1ba079807eeaf795a 2023-03-09 01:55:26,650 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/72f56cdcf0b942a1ba079807eeaf795a, entries=3, sequenceid=24, filesize=5.0 K 2023-03-09 01:55:26,650 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/00c7e796853b4a5e9b3384925b442817 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/00c7e796853b4a5e9b3384925b442817 2023-03-09 01:55:26,655 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00c7e796853b4a5e9b3384925b442817 2023-03-09 01:55:26,655 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/00c7e796853b4a5e9b3384925b442817, entries=1, sequenceid=24, filesize=5.0 K 2023-03-09 01:55:26,655 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 18e9a9ff9da4b5322903ba24ba13b2e1 in 851ms, sequenceid=24, compaction requested=true 2023-03-09 01:55:26,656 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:26,656 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:26,656 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:27,043 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/928df6ee6acd441aaeb3214409e752af 2023-03-09 01:55:27,047 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 928df6ee6acd441aaeb3214409e752af 2023-03-09 01:55:27,048 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/68c9f8cd8fa04baeaffaa91df581cd38 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/68c9f8cd8fa04baeaffaa91df581cd38 2023-03-09 01:55:27,051 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68c9f8cd8fa04baeaffaa91df581cd38 2023-03-09 01:55:27,051 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/68c9f8cd8fa04baeaffaa91df581cd38, entries=3, sequenceid=24, filesize=5.0 K 2023-03-09 01:55:27,052 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/928df6ee6acd441aaeb3214409e752af as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/928df6ee6acd441aaeb3214409e752af 2023-03-09 01:55:27,055 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 928df6ee6acd441aaeb3214409e752af 2023-03-09 01:55:27,055 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/928df6ee6acd441aaeb3214409e752af, entries=1, sequenceid=24, filesize=5.0 K 2023-03-09 01:55:27,056 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 00f768e6a69add6056570cdcb3ad746e in 1250ms, sequenceid=24, compaction requested=true 2023-03-09 01:55:27,056 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:27,056 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:27,056 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:27,440 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/8767e74614c64aeaaf34f2a5655e264e 2023-03-09 01:55:27,444 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8767e74614c64aeaaf34f2a5655e264e 2023-03-09 01:55:27,445 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/f7ffccdc37e24c36926633d1d4c0a3dc as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/f7ffccdc37e24c36926633d1d4c0a3dc 2023-03-09 01:55:27,449 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7ffccdc37e24c36926633d1d4c0a3dc 2023-03-09 01:55:27,449 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/f7ffccdc37e24c36926633d1d4c0a3dc, entries=3, sequenceid=24, filesize=5.0 K 2023-03-09 01:55:27,449 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/8767e74614c64aeaaf34f2a5655e264e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/8767e74614c64aeaaf34f2a5655e264e 2023-03-09 01:55:27,453 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8767e74614c64aeaaf34f2a5655e264e 2023-03-09 01:55:27,454 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/8767e74614c64aeaaf34f2a5655e264e, entries=1, sequenceid=24, filesize=5.0 K 2023-03-09 01:55:27,455 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for abbc0d228ef2735615e5e8f7e1c367fc in 1650ms, sequenceid=24, compaction requested=true 2023-03-09 01:55:27,455 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:27,455 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:27,455 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:27,455 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:27,455 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,455 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-09 01:55:27,455 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-09 01:55:27,457 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:27,457 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:27,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:27,457 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:27,457 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:27,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:27,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:27,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:27,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:27,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-09 01:55:27,459 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-09 01:55:27,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d97ab19[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:27,459 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:27,463 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:27,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:27,463 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-09 01:55:27,463 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-09 01:55:27,463 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:27,463 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,463 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:27,465 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-09 01:55:27,465 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:27,465 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,465 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-09 01:55:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:27,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:27,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-09 01:55:27,467 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-09 01:55:27,467 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:27,467 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:27,467 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-09 01:55:27,467 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:27,468 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:27,468 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:27,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:27,468 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:27,469 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:27,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:27,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:27,470 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:27,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-09 01:55:27,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:27,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:27,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-09 01:55:27,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:27,477 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:27,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-09 01:55:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-09 01:55:27,478 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-09 01:55:27,477 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:27,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:27,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:27,578 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:27,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:27,579 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:27,579 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): abbc0d228ef2735615e5e8f7e1c367fc/family1 is initiating major compaction (all files) 2023-03-09 01:55:27,580 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:27,580 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:27,580 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of abbc0d228ef2735615e5e8f7e1c367fc/family1 in testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:27,581 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/554613f5dee54006ad0ca4cc98415b67, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a19058eff3f84d83a01b0191d4a9958d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/c03f67bd5eda426d98fe2d2f23cddf3c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/9a43c7ed86e441a396bd23845a381994, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/f7ffccdc37e24c36926633d1d4c0a3dc] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp, totalSize=26.7 K 2023-03-09 01:55:27,581 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 554613f5dee54006ad0ca4cc98415b67, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916810 2023-03-09 01:55:27,582 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting a19058eff3f84d83a01b0191d4a9958d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:27,582 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting c03f67bd5eda426d98fe2d2f23cddf3c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:27,582 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 9a43c7ed86e441a396bd23845a381994, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921802 2023-03-09 01:55:27,582 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting f7ffccdc37e24c36926633d1d4c0a3dc, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-09 01:55:27,600 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): abbc0d228ef2735615e5e8f7e1c367fc#family1#compaction#107 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:28,008 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230309082485e9c59941c1a4a26a5f6935f8a6_del 2023-03-09 01:55:28,008 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309082485e9c59941c1a4a26a5f6935f8a6_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309082485e9c59941c1a4a26a5f6935f8a6_del 2023-03-09 01:55:28,822 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7bbcac145c534ba8bc25c71081434654 2023-03-09 01:55:28,823 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/7bbcac145c534ba8bc25c71081434654 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/7bbcac145c534ba8bc25c71081434654 2023-03-09 01:55:28,826 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7bbcac145c534ba8bc25c71081434654 2023-03-09 01:55:28,827 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in abbc0d228ef2735615e5e8f7e1c367fc/family1 of abbc0d228ef2735615e5e8f7e1c367fc into 7bbcac145c534ba8bc25c71081434654(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:28,827 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:28,827 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:28,827 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): abbc0d228ef2735615e5e8f7e1c367fc/family2 is initiating major compaction (all files) 2023-03-09 01:55:28,827 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:28,827 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:28,827 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of abbc0d228ef2735615e5e8f7e1c367fc/family2 in testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:28,828 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/def1134fc58a4a269289ed9ac51b532e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/e23c8b4c586a48068d85ad2125091e46, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/f626b340147b41f481b99eceb82737c3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/5c406707c2a64cd4ab677f3124d6af50, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/8767e74614c64aeaaf34f2a5655e264e] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp, totalSize=25.4 K 2023-03-09 01:55:28,828 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting def1134fc58a4a269289ed9ac51b532e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916810 2023-03-09 01:55:28,828 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting e23c8b4c586a48068d85ad2125091e46, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:28,829 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting f626b340147b41f481b99eceb82737c3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:28,829 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 5c406707c2a64cd4ab677f3124d6af50, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921802 2023-03-09 01:55:28,829 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 8767e74614c64aeaaf34f2a5655e264e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-09 01:55:28,840 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): abbc0d228ef2735615e5e8f7e1c367fc#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:28,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/554613f5dee54006ad0ca4cc98415b67, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a19058eff3f84d83a01b0191d4a9958d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/c03f67bd5eda426d98fe2d2f23cddf3c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/9a43c7ed86e441a396bd23845a381994, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/f7ffccdc37e24c36926633d1d4c0a3dc] to archive 2023-03-09 01:55:28,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:28,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/554613f5dee54006ad0ca4cc98415b67 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/554613f5dee54006ad0ca4cc98415b67 2023-03-09 01:55:28,857 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a19058eff3f84d83a01b0191d4a9958d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a19058eff3f84d83a01b0191d4a9958d 2023-03-09 01:55:28,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/c03f67bd5eda426d98fe2d2f23cddf3c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/c03f67bd5eda426d98fe2d2f23cddf3c 2023-03-09 01:55:28,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/9a43c7ed86e441a396bd23845a381994 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/9a43c7ed86e441a396bd23845a381994 2023-03-09 01:55:28,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/f7ffccdc37e24c36926633d1d4c0a3dc to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/f7ffccdc37e24c36926633d1d4c0a3dc 2023-03-09 01:55:29,249 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303090c6d7a77dbdd4150a39556745b75e1db_del 2023-03-09 01:55:29,249 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303090c6d7a77dbdd4150a39556745b75e1db_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303090c6d7a77dbdd4150a39556745b75e1db_del 2023-03-09 01:55:30,062 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db13546a2a52419da91ada0616887f46 2023-03-09 01:55:30,063 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/db13546a2a52419da91ada0616887f46 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/db13546a2a52419da91ada0616887f46 2023-03-09 01:55:30,067 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db13546a2a52419da91ada0616887f46 2023-03-09 01:55:30,068 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in abbc0d228ef2735615e5e8f7e1c367fc/family2 of abbc0d228ef2735615e5e8f7e1c367fc into db13546a2a52419da91ada0616887f46(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:30,068 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:30,068 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:30,068 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 18e9a9ff9da4b5322903ba24ba13b2e1/family1 is initiating major compaction (all files) 2023-03-09 01:55:30,068 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:30,069 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:30,069 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 18e9a9ff9da4b5322903ba24ba13b2e1/family1 in testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:30,069 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/6b9df6dda32a470bb7beedc493b4d7c2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/46f048b3bec44725bbd2a2dd1547ad65, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/e52084fb3ce043d3b3193b82c7c8c28d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/c092b1b2898b430cb5a6fd6ef235b622, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/72f56cdcf0b942a1ba079807eeaf795a] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp, totalSize=26.7 K 2023-03-09 01:55:30,069 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 6b9df6dda32a470bb7beedc493b4d7c2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916809 2023-03-09 01:55:30,069 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 46f048b3bec44725bbd2a2dd1547ad65, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:30,070 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting e52084fb3ce043d3b3193b82c7c8c28d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:30,070 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting c092b1b2898b430cb5a6fd6ef235b622, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921801 2023-03-09 01:55:30,070 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 72f56cdcf0b942a1ba079807eeaf795a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-09 01:55:30,082 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 18e9a9ff9da4b5322903ba24ba13b2e1#family1#compaction#109 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:30,155 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/def1134fc58a4a269289ed9ac51b532e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/e23c8b4c586a48068d85ad2125091e46, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/f626b340147b41f481b99eceb82737c3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/5c406707c2a64cd4ab677f3124d6af50, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/8767e74614c64aeaaf34f2a5655e264e] to archive 2023-03-09 01:55:30,156 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:30,158 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/def1134fc58a4a269289ed9ac51b532e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/def1134fc58a4a269289ed9ac51b532e 2023-03-09 01:55:30,159 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/e23c8b4c586a48068d85ad2125091e46 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/e23c8b4c586a48068d85ad2125091e46 2023-03-09 01:55:30,160 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/f626b340147b41f481b99eceb82737c3 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/f626b340147b41f481b99eceb82737c3 2023-03-09 01:55:30,161 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/5c406707c2a64cd4ab677f3124d6af50 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/5c406707c2a64cd4ab677f3124d6af50 2023-03-09 01:55:30,162 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/8767e74614c64aeaaf34f2a5655e264e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/8767e74614c64aeaaf34f2a5655e264e 2023-03-09 01:55:30,490 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230309501b8d8fb8fd4904bfe69c3b3f561366_del 2023-03-09 01:55:30,490 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309501b8d8fb8fd4904bfe69c3b3f561366_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309501b8d8fb8fd4904bfe69c3b3f561366_del 2023-03-09 01:55:31,305 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 080b00be8f2745859f51a21f5974bc6e 2023-03-09 01:55:31,306 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/080b00be8f2745859f51a21f5974bc6e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/080b00be8f2745859f51a21f5974bc6e 2023-03-09 01:55:31,309 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 080b00be8f2745859f51a21f5974bc6e 2023-03-09 01:55:31,310 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 18e9a9ff9da4b5322903ba24ba13b2e1/family1 of 18e9a9ff9da4b5322903ba24ba13b2e1 into 080b00be8f2745859f51a21f5974bc6e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:31,310 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:31,310 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:31,310 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 18e9a9ff9da4b5322903ba24ba13b2e1/family2 is initiating major compaction (all files) 2023-03-09 01:55:31,311 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:31,311 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:31,311 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 18e9a9ff9da4b5322903ba24ba13b2e1/family2 in testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:31,311 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/60bbe8792b3d43189452c94ff064687b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/647e37787cd149a190171dfdcdb08187, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/e8e622fe17cc4feaaa821052b05b2f1d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/0a6c637662964e48a5d1d38c97d82cef, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/00c7e796853b4a5e9b3384925b442817] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp, totalSize=25.4 K 2023-03-09 01:55:31,311 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 60bbe8792b3d43189452c94ff064687b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916809 2023-03-09 01:55:31,312 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 647e37787cd149a190171dfdcdb08187, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:31,312 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting e8e622fe17cc4feaaa821052b05b2f1d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:31,312 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 0a6c637662964e48a5d1d38c97d82cef, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921801 2023-03-09 01:55:31,312 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 00c7e796853b4a5e9b3384925b442817, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-09 01:55:31,322 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 18e9a9ff9da4b5322903ba24ba13b2e1#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:31,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/6b9df6dda32a470bb7beedc493b4d7c2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/46f048b3bec44725bbd2a2dd1547ad65, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/e52084fb3ce043d3b3193b82c7c8c28d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/c092b1b2898b430cb5a6fd6ef235b622, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/72f56cdcf0b942a1ba079807eeaf795a] to archive 2023-03-09 01:55:31,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:31,355 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/6b9df6dda32a470bb7beedc493b4d7c2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/6b9df6dda32a470bb7beedc493b4d7c2 2023-03-09 01:55:31,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/46f048b3bec44725bbd2a2dd1547ad65 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/46f048b3bec44725bbd2a2dd1547ad65 2023-03-09 01:55:31,357 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/e52084fb3ce043d3b3193b82c7c8c28d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/e52084fb3ce043d3b3193b82c7c8c28d 2023-03-09 01:55:31,357 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/c092b1b2898b430cb5a6fd6ef235b622 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/c092b1b2898b430cb5a6fd6ef235b622 2023-03-09 01:55:31,358 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/72f56cdcf0b942a1ba079807eeaf795a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/72f56cdcf0b942a1ba079807eeaf795a 2023-03-09 01:55:31,732 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030924ba2d9bd7bb45c4b1ceda80e2874964_del 2023-03-09 01:55:31,733 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030924ba2d9bd7bb45c4b1ceda80e2874964_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030924ba2d9bd7bb45c4b1ceda80e2874964_del 2023-03-09 01:55:32,547 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 572144be737042b4b014d1666d23d354 2023-03-09 01:55:32,547 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/572144be737042b4b014d1666d23d354 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/572144be737042b4b014d1666d23d354 2023-03-09 01:55:32,551 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 572144be737042b4b014d1666d23d354 2023-03-09 01:55:32,552 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 18e9a9ff9da4b5322903ba24ba13b2e1/family2 of 18e9a9ff9da4b5322903ba24ba13b2e1 into 572144be737042b4b014d1666d23d354(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:32,552 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:32,552 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:32,552 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 00f768e6a69add6056570cdcb3ad746e/family1 is initiating major compaction (all files) 2023-03-09 01:55:32,552 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:32,552 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:32,553 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 00f768e6a69add6056570cdcb3ad746e/family1 in testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:32,553 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/bd7eaca3d0bb49c881093d8bffbe4a61, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/9f28ae193d6b4e34bbb73a959c56c0f2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/82dcda8270744c87b9890f8a8573eb1a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/c8a9da0c9dea4201985e0dda7f99ce78, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/68c9f8cd8fa04baeaffaa91df581cd38] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp, totalSize=26.7 K 2023-03-09 01:55:32,554 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting bd7eaca3d0bb49c881093d8bffbe4a61, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916808 2023-03-09 01:55:32,554 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 9f28ae193d6b4e34bbb73a959c56c0f2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:32,555 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 82dcda8270744c87b9890f8a8573eb1a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:32,555 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting c8a9da0c9dea4201985e0dda7f99ce78, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921801 2023-03-09 01:55:32,556 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 68c9f8cd8fa04baeaffaa91df581cd38, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-09 01:55:32,567 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 00f768e6a69add6056570cdcb3ad746e#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:32,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/60bbe8792b3d43189452c94ff064687b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/647e37787cd149a190171dfdcdb08187, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/e8e622fe17cc4feaaa821052b05b2f1d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/0a6c637662964e48a5d1d38c97d82cef, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/00c7e796853b4a5e9b3384925b442817] to archive 2023-03-09 01:55:32,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:32,654 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/60bbe8792b3d43189452c94ff064687b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/60bbe8792b3d43189452c94ff064687b 2023-03-09 01:55:32,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/647e37787cd149a190171dfdcdb08187 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/647e37787cd149a190171dfdcdb08187 2023-03-09 01:55:32,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/e8e622fe17cc4feaaa821052b05b2f1d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/e8e622fe17cc4feaaa821052b05b2f1d 2023-03-09 01:55:32,658 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/0a6c637662964e48a5d1d38c97d82cef to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/0a6c637662964e48a5d1d38c97d82cef 2023-03-09 01:55:32,659 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/00c7e796853b4a5e9b3384925b442817 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/00c7e796853b4a5e9b3384925b442817 2023-03-09 01:55:32,975 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309aa9545f9c41141e2838a2116065362ab_del 2023-03-09 01:55:32,975 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309aa9545f9c41141e2838a2116065362ab_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309aa9545f9c41141e2838a2116065362ab_del 2023-03-09 01:55:33,789 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fceb5fa8161f4343a26f6d83b908baeb 2023-03-09 01:55:33,790 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/fceb5fa8161f4343a26f6d83b908baeb as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/fceb5fa8161f4343a26f6d83b908baeb 2023-03-09 01:55:33,794 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fceb5fa8161f4343a26f6d83b908baeb 2023-03-09 01:55:33,795 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 00f768e6a69add6056570cdcb3ad746e/family1 of 00f768e6a69add6056570cdcb3ad746e into fceb5fa8161f4343a26f6d83b908baeb(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:33,795 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:33,795 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:33,795 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 00f768e6a69add6056570cdcb3ad746e/family2 is initiating major compaction (all files) 2023-03-09 01:55:33,795 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:33,795 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:33,796 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 00f768e6a69add6056570cdcb3ad746e/family2 in testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:33,796 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/51a937efd2f644babe8351499c9fa15a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/b54f40cf492f48f2a959cbed0a4fd66d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/8c204b7188634b12a4b18c8cb9e96815, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/6fd2b3985d2b4c7b812d559361310f38, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/928df6ee6acd441aaeb3214409e752af] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp, totalSize=25.4 K 2023-03-09 01:55:33,797 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 51a937efd2f644babe8351499c9fa15a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916808 2023-03-09 01:55:33,797 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting b54f40cf492f48f2a959cbed0a4fd66d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:33,798 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 8c204b7188634b12a4b18c8cb9e96815, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:33,798 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 6fd2b3985d2b4c7b812d559361310f38, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921801 2023-03-09 01:55:33,799 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 928df6ee6acd441aaeb3214409e752af, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-09 01:55:33,815 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 00f768e6a69add6056570cdcb3ad746e#family2#compaction#112 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:33,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/bd7eaca3d0bb49c881093d8bffbe4a61, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/9f28ae193d6b4e34bbb73a959c56c0f2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/82dcda8270744c87b9890f8a8573eb1a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/c8a9da0c9dea4201985e0dda7f99ce78, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/68c9f8cd8fa04baeaffaa91df581cd38] to archive 2023-03-09 01:55:33,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:33,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/bd7eaca3d0bb49c881093d8bffbe4a61 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/bd7eaca3d0bb49c881093d8bffbe4a61 2023-03-09 01:55:33,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/9f28ae193d6b4e34bbb73a959c56c0f2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/9f28ae193d6b4e34bbb73a959c56c0f2 2023-03-09 01:55:33,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/82dcda8270744c87b9890f8a8573eb1a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/82dcda8270744c87b9890f8a8573eb1a 2023-03-09 01:55:33,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/c8a9da0c9dea4201985e0dda7f99ce78 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/c8a9da0c9dea4201985e0dda7f99ce78 2023-03-09 01:55:33,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/68c9f8cd8fa04baeaffaa91df581cd38 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/68c9f8cd8fa04baeaffaa91df581cd38 2023-03-09 01:55:34,224 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309584d72e480fb46e89925dd5811a9e8e7_del 2023-03-09 01:55:34,224 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309584d72e480fb46e89925dd5811a9e8e7_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309584d72e480fb46e89925dd5811a9e8e7_del 2023-03-09 01:55:35,039 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a86db3fc8d8c42e69269f5c7f5c31c00 2023-03-09 01:55:35,040 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/a86db3fc8d8c42e69269f5c7f5c31c00 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/a86db3fc8d8c42e69269f5c7f5c31c00 2023-03-09 01:55:35,044 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a86db3fc8d8c42e69269f5c7f5c31c00 2023-03-09 01:55:35,045 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 00f768e6a69add6056570cdcb3ad746e/family2 of 00f768e6a69add6056570cdcb3ad746e into a86db3fc8d8c42e69269f5c7f5c31c00(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:35,045 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:35,049 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/51a937efd2f644babe8351499c9fa15a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/b54f40cf492f48f2a959cbed0a4fd66d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/8c204b7188634b12a4b18c8cb9e96815, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/6fd2b3985d2b4c7b812d559361310f38, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/928df6ee6acd441aaeb3214409e752af] to archive 2023-03-09 01:55:35,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:35,051 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:35,053 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/51a937efd2f644babe8351499c9fa15a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/51a937efd2f644babe8351499c9fa15a 2023-03-09 01:55:35,055 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/b54f40cf492f48f2a959cbed0a4fd66d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/b54f40cf492f48f2a959cbed0a4fd66d 2023-03-09 01:55:35,055 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:35,057 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/8c204b7188634b12a4b18c8cb9e96815 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/8c204b7188634b12a4b18c8cb9e96815 2023-03-09 01:55:35,060 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/6fd2b3985d2b4c7b812d559361310f38 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/6fd2b3985d2b4c7b812d559361310f38 2023-03-09 01:55:35,062 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/928df6ee6acd441aaeb3214409e752af to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/928df6ee6acd441aaeb3214409e752af 2023-03-09 01:55:35,064 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:35,065 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:35,072 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-09 01:55:35,107 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-09 01:55:35,110 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309aa9545f9c41141e2838a2116065362ab_del 2023-03-09 01:55:35,113 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230309501b8d8fb8fd4904bfe69c3b3f561366_del 2023-03-09 01:55:35,116 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230309082485e9c59941c1a4a26a5f6935f8a6_del 2023-03-09 01:55:35,116 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:35,117 DEBUG [Listener at localhost/33973] zookeeper.ReadOnlyZKClient(139): Connect 0x5289509b to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:35,122 DEBUG [Listener at localhost/33973] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3eac4e78, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:35,122 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:35,122 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:35,122 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:35,588 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303093de2db0a7c6b4ec6b7e7280f8aec9b5b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303093de2db0a7c6b4ec6b7e7280f8aec9b5b 2023-03-09 01:55:35,589 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:35,608 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:55:35,612 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47518, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:55:35,619 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:55:35,622 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41972, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:55:35,634 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309/family1/a8ac15dc4afb4148b07256a8fa8d56e7 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-09 01:55:35,635 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309/family1/a8ac15dc4afb4148b07256a8fa8d56e7}] 2023-03-09 01:55:35,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309/family1/a8ac15dc4afb4148b07256a8fa8d56e7 for inclusion in 18e9a9ff9da4b5322903ba24ba13b2e1/family1 2023-03-09 01:55:35,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-09 01:55:35,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-09 01:55:35,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:35,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309/family1/a8ac15dc4afb4148b07256a8fa8d56e7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__biccbcjfon23aba97hrl7g2500jhaisave5361j17r0vetnbp5l6i2kl670iarlp/family1/a8ac15dc4afb4148b07256a8fa8d56e7 2023-03-09 01:55:35,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__biccbcjfon23aba97hrl7g2500jhaisave5361j17r0vetnbp5l6i2kl670iarlp/family1/a8ac15dc4afb4148b07256a8fa8d56e7 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/b764918ae73241b3bdbd646f18e6e4a2_SeqId_29_ 2023-03-09 01:55:35,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__biccbcjfon23aba97hrl7g2500jhaisave5361j17r0vetnbp5l6i2kl670iarlp/family1/a8ac15dc4afb4148b07256a8fa8d56e7 into 18e9a9ff9da4b5322903ba24ba13b2e1/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/b764918ae73241b3bdbd646f18e6e4a2_SeqId_29_ - updating store file list. 2023-03-09 01:55:35,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/b764918ae73241b3bdbd646f18e6e4a2_SeqId_29_ into 18e9a9ff9da4b5322903ba24ba13b2e1/family1 2023-03-09 01:55:35,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__biccbcjfon23aba97hrl7g2500jhaisave5361j17r0vetnbp5l6i2kl670iarlp/family1/a8ac15dc4afb4148b07256a8fa8d56e7 into 18e9a9ff9da4b5322903ba24ba13b2e1/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/b764918ae73241b3bdbd646f18e6e4a2_SeqId_29_) 2023-03-09 01:55:35,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__biccbcjfon23aba97hrl7g2500jhaisave5361j17r0vetnbp5l6i2kl670iarlp/family1/a8ac15dc4afb4148b07256a8fa8d56e7 2023-03-09 01:55:35,651 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:35,652 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:35,653 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7 2023-03-09 01:55:35,654 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84 2023-03-09 01:55:35,655 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b 2023-03-09 01:55:35,656 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99 2023-03-09 01:55:35,656 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-09 01:55:35,656 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309' 2023-03-09 01:55:35,657 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:35,980 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309a51a7c9ebf2b48a4962233b0b0c2b438 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309a51a7c9ebf2b48a4962233b0b0c2b438 2023-03-09 01:55:35,981 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:35,989 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309c026565af9bd49bd801e226238db7596 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309c026565af9bd49bd801e226238db7596 2023-03-09 01:55:35,990 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:36,007 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309/family1/d79de29d124041e7baca3a9d4ae7ca42 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-09 01:55:36,009 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309/family1/d79de29d124041e7baca3a9d4ae7ca42}] 2023-03-09 01:55:36,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309/family1/d79de29d124041e7baca3a9d4ae7ca42 for inclusion in 00f768e6a69add6056570cdcb3ad746e/family1 2023-03-09 01:55:36,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-09 01:55:36,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last=1 2023-03-09 01:55:36,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:36,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309/family1/d79de29d124041e7baca3a9d4ae7ca42 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__2cbm5p1rts6ig3f90rtfc489i6hh4cc0a61e9l4uf2rv61ri847qf1ffukfhrjrr/family1/d79de29d124041e7baca3a9d4ae7ca42 2023-03-09 01:55:36,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__2cbm5p1rts6ig3f90rtfc489i6hh4cc0a61e9l4uf2rv61ri847qf1ffukfhrjrr/family1/d79de29d124041e7baca3a9d4ae7ca42 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/446e175e02de45f09f63b31077a470c2_SeqId_29_ 2023-03-09 01:55:36,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__2cbm5p1rts6ig3f90rtfc489i6hh4cc0a61e9l4uf2rv61ri847qf1ffukfhrjrr/family1/d79de29d124041e7baca3a9d4ae7ca42 into 00f768e6a69add6056570cdcb3ad746e/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/446e175e02de45f09f63b31077a470c2_SeqId_29_ - updating store file list. 2023-03-09 01:55:36,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/446e175e02de45f09f63b31077a470c2_SeqId_29_ into 00f768e6a69add6056570cdcb3ad746e/family1 2023-03-09 01:55:36,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__2cbm5p1rts6ig3f90rtfc489i6hh4cc0a61e9l4uf2rv61ri847qf1ffukfhrjrr/family1/d79de29d124041e7baca3a9d4ae7ca42 into 00f768e6a69add6056570cdcb3ad746e/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/446e175e02de45f09f63b31077a470c2_SeqId_29_) 2023-03-09 01:55:36,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__2cbm5p1rts6ig3f90rtfc489i6hh4cc0a61e9l4uf2rv61ri847qf1ffukfhrjrr/family1/d79de29d124041e7baca3a9d4ae7ca42 2023-03-09 01:55:36,028 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:36,029 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:36,031 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f 2023-03-09 01:55:36,033 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5 2023-03-09 01:55:36,034 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309/family1/573c5861e88841989fc6853814446951 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-09 01:55:36,035 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914 2023-03-09 01:55:36,035 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309/family1/573c5861e88841989fc6853814446951}] 2023-03-09 01:55:36,036 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412 2023-03-09 01:55:36,036 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-09 01:55:36,036 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309' 2023-03-09 01:55:36,038 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:36,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309/family1/573c5861e88841989fc6853814446951 for inclusion in abbc0d228ef2735615e5e8f7e1c367fc/family1 2023-03-09 01:55:36,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-09 01:55:36,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-09 01:55:36,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:36,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309/family1/573c5861e88841989fc6853814446951 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ststm39h41t42jmg79bqeohm6bcm312l0f7gatpcfgnkfd9sgemk98514tl0ervs/family1/573c5861e88841989fc6853814446951 2023-03-09 01:55:36,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ststm39h41t42jmg79bqeohm6bcm312l0f7gatpcfgnkfd9sgemk98514tl0ervs/family1/573c5861e88841989fc6853814446951 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/2c95bc2ca9ad4805b61f6e90f117bcb8_SeqId_29_ 2023-03-09 01:55:36,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ststm39h41t42jmg79bqeohm6bcm312l0f7gatpcfgnkfd9sgemk98514tl0ervs/family1/573c5861e88841989fc6853814446951 into abbc0d228ef2735615e5e8f7e1c367fc/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/2c95bc2ca9ad4805b61f6e90f117bcb8_SeqId_29_ - updating store file list. 2023-03-09 01:55:36,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/2c95bc2ca9ad4805b61f6e90f117bcb8_SeqId_29_ into abbc0d228ef2735615e5e8f7e1c367fc/family1 2023-03-09 01:55:36,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ststm39h41t42jmg79bqeohm6bcm312l0f7gatpcfgnkfd9sgemk98514tl0ervs/family1/573c5861e88841989fc6853814446951 into abbc0d228ef2735615e5e8f7e1c367fc/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/2c95bc2ca9ad4805b61f6e90f117bcb8_SeqId_29_) 2023-03-09 01:55:36,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ststm39h41t42jmg79bqeohm6bcm312l0f7gatpcfgnkfd9sgemk98514tl0ervs/family1/573c5861e88841989fc6853814446951 2023-03-09 01:55:36,051 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:36,052 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:36,053 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa 2023-03-09 01:55:36,054 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178 2023-03-09 01:55:36,055 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96 2023-03-09 01:55:36,056 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763 2023-03-09 01:55:36,056 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-09 01:55:36,056 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309' 2023-03-09 01:55:36,056 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:36,056 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:36,056 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:36,056 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309aa9545f9c41141e2838a2116065362ab_del] 2023-03-09 01:55:36,057 DEBUG [Listener at localhost/33973] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:36,058 DEBUG [Listener at localhost/33973] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309aa9545f9c41141e2838a2116065362ab_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309aa9545f9c41141e2838a2116065362ab_del 2023-03-09 01:55:36,058 INFO [Listener at localhost/33973] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-09 01:55:36,058 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309501b8d8fb8fd4904bfe69c3b3f561366_del] 2023-03-09 01:55:36,059 DEBUG [Listener at localhost/33973] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:36,060 DEBUG [Listener at localhost/33973] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309501b8d8fb8fd4904bfe69c3b3f561366_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230309501b8d8fb8fd4904bfe69c3b3f561366_del 2023-03-09 01:55:36,060 INFO [Listener at localhost/33973] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-09 01:55:36,060 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309082485e9c59941c1a4a26a5f6935f8a6_del] 2023-03-09 01:55:36,061 DEBUG [Listener at localhost/33973] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:36,062 DEBUG [Listener at localhost/33973] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309082485e9c59941c1a4a26a5f6935f8a6_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309082485e9c59941c1a4a26a5f6935f8a6_del 2023-03-09 01:55:36,062 INFO [Listener at localhost/33973] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-09 01:55:36,065 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:36,067 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:36,070 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:36,071 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:36,077 INFO [Listener at localhost/33973] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-09 01:55:36,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-09 01:55:36,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-09 01:55:36,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-09 01:55:36,081 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326936081"}]},"ts":"1678326936081"} 2023-03-09 01:55:36,082 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-09 01:55:36,083 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-09 01:55:36,084 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, UNASSIGN}] 2023-03-09 01:55:36,086 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, UNASSIGN 2023-03-09 01:55:36,086 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, UNASSIGN 2023-03-09 01:55:36,086 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, UNASSIGN 2023-03-09 01:55:36,086 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=00f768e6a69add6056570cdcb3ad746e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:36,086 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=18e9a9ff9da4b5322903ba24ba13b2e1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:36,086 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=abbc0d228ef2735615e5e8f7e1c367fc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:36,086 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326936086"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326936086"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326936086"}]},"ts":"1678326936086"} 2023-03-09 01:55:36,086 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326936086"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326936086"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326936086"}]},"ts":"1678326936086"} 2023-03-09 01:55:36,087 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326936086"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326936086"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326936086"}]},"ts":"1678326936086"} 2023-03-09 01:55:36,088 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure 00f768e6a69add6056570cdcb3ad746e, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:36,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 18e9a9ff9da4b5322903ba24ba13b2e1, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:36,089 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure abbc0d228ef2735615e5e8f7e1c367fc, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:36,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-09 01:55:36,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:36,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 00f768e6a69add6056570cdcb3ad746e, disabling compactions & flushes 2023-03-09 01:55:36,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:36,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:36,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. after waiting 0 ms 2023-03-09 01:55:36,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:36,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-09 01:55:36,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:36,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:36,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:36,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:36,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 18e9a9ff9da4b5322903ba24ba13b2e1, disabling compactions & flushes 2023-03-09 01:55:36,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:36,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:36,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. after waiting 0 ms 2023-03-09 01:55:36,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:36,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=00f768e6a69add6056570cdcb3ad746e, regionState=CLOSED 2023-03-09 01:55:36,250 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326936250"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326936250"}]},"ts":"1678326936250"} 2023-03-09 01:55:36,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=89 2023-03-09 01:55:36,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=89, state=SUCCESS; CloseRegionProcedure 00f768e6a69add6056570cdcb3ad746e, server=jenkins-hbase4.apache.org,46511,1678326819757 in 165 msec 2023-03-09 01:55:36,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-09 01:55:36,255 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, UNASSIGN in 170 msec 2023-03-09 01:55:36,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:36,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:36,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:36,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:36,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing abbc0d228ef2735615e5e8f7e1c367fc, disabling compactions & flushes 2023-03-09 01:55:36,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:36,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:36,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. after waiting 0 ms 2023-03-09 01:55:36,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:36,257 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=18e9a9ff9da4b5322903ba24ba13b2e1, regionState=CLOSED 2023-03-09 01:55:36,257 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326936257"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326936257"}]},"ts":"1678326936257"} 2023-03-09 01:55:36,261 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-03-09 01:55:36,261 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure 18e9a9ff9da4b5322903ba24ba13b2e1, server=jenkins-hbase4.apache.org,46511,1678326819757 in 172 msec 2023-03-09 01:55:36,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-09 01:55:36,262 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, UNASSIGN in 177 msec 2023-03-09 01:55:36,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:36,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:36,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:36,264 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=abbc0d228ef2735615e5e8f7e1c367fc, regionState=CLOSED 2023-03-09 01:55:36,265 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326936264"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326936264"}]},"ts":"1678326936264"} 2023-03-09 01:55:36,266 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-03-09 01:55:36,266 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure abbc0d228ef2735615e5e8f7e1c367fc, server=jenkins-hbase4.apache.org,46511,1678326819757 in 177 msec 2023-03-09 01:55:36,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=88 2023-03-09 01:55:36,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, UNASSIGN in 182 msec 2023-03-09 01:55:36,268 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326936268"}]},"ts":"1678326936268"} 2023-03-09 01:55:36,269 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-09 01:55:36,270 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-09 01:55:36,272 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 194 msec 2023-03-09 01:55:36,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-09 01:55:36,382 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-09 01:55:36,392 INFO [Listener at localhost/33973] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1678326923611-1678326936392 2023-03-09 01:55:36,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-09 01:55:36,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1678326936393 (current time:1678326936393). 2023-03-09 01:55:36,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-09 01:55:36,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] zookeeper.ReadOnlyZKClient(139): Connect 0x74541e98 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:36,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5063a7d8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:36,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:55:36,402 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47528, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:55:36,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x74541e98 to 127.0.0.1:50548 2023-03-09 01:55:36,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:55:36,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-09 01:55:36,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-09 01:55:36,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-09 01:55:36,406 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-09 01:55:36,407 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1678327536407 2023-03-09 01:55:36,407 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1678326923611-1678326936392 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-09 01:55:36,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-09 01:55:36,407 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:36,407 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-09 01:55:36,507 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:36,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:36,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-09 01:55:36,509 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-09 01:55:36,709 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:36,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:36,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-09 01:55:36,710 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-09 01:55:36,813 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-09 01:55:36,815 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-09 01:55:36,815 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-09 01:55:36,815 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-09 01:55:36,815 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-09 01:55:36,815 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-09 01:55:36,815 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-09 01:55:36,815 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-09 01:55:36,815 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/2c95bc2ca9ad4805b61f6e90f117bcb8_SeqId_29_, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/7bbcac145c534ba8bc25c71081434654] hfiles 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/446e175e02de45f09f63b31077a470c2_SeqId_29_, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/fceb5fa8161f4343a26f6d83b908baeb] hfiles 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/080b00be8f2745859f51a21f5974bc6e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/b764918ae73241b3bdbd646f18e6e4a2_SeqId_29_] hfiles 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/2c95bc2ca9ad4805b61f6e90f117bcb8_SeqId_29_ 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303093de2db0a7c6b4ec6b7e7280f8aec9b5b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309c026565af9bd49bd801e226238db7596, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309a51a7c9ebf2b48a4962233b0b0c2b438] hfiles 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/7bbcac145c534ba8bc25c71081434654 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/080b00be8f2745859f51a21f5974bc6e 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/446e175e02de45f09f63b31077a470c2_SeqId_29_ 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/b764918ae73241b3bdbd646f18e6e4a2_SeqId_29_ 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/fceb5fa8161f4343a26f6d83b908baeb 2023-03-09 01:55:36,818 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303093de2db0a7c6b4ec6b7e7280f8aec9b5b 2023-03-09 01:55:36,819 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309c026565af9bd49bd801e226238db7596 2023-03-09 01:55:36,819 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309a51a7c9ebf2b48a4962233b0b0c2b438 2023-03-09 01:55:36,819 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/db13546a2a52419da91ada0616887f46] hfiles 2023-03-09 01:55:36,819 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/572144be737042b4b014d1666d23d354] hfiles 2023-03-09 01:55:36,819 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/db13546a2a52419da91ada0616887f46 2023-03-09 01:55:36,819 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/a86db3fc8d8c42e69269f5c7f5c31c00] hfiles 2023-03-09 01:55:36,819 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/572144be737042b4b014d1666d23d354 2023-03-09 01:55:36,819 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/a86db3fc8d8c42e69269f5c7f5c31c00 2023-03-09 01:55:36,827 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030924ba2d9bd7bb45c4b1ceda80e2874964_del, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030927a1a49fb1f04db4bb45b3beaecf8dc6, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309b7ed83e86ebb480fbb10faee5ca824c1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309beb0560b7a674037913ac15ba3bb037f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309fec823224a2f4e7abc862491640daf44, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303090c6d7a77dbdd4150a39556745b75e1db_del, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309aa9b7d5917264d269dc6eeff4e21a1c5, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309bb4349f7ac904ce084506bf4674e0c82, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309e0f7092228534ccabbe807808edc1f89, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309ee66da52298848a4a316e031c7adb822, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303090f8e236dd2b44d0aa74f234ad0abaf0b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309251d64456774424d9d2445a93d454feb, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303093df924d792694c629ae9256c3bc125ef, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309584d72e480fb46e89925dd5811a9e8e7_del, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309b28a61ad448d408b804acac511dc7db0] hfiles 2023-03-09 01:55:36,827 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030924ba2d9bd7bb45c4b1ceda80e2874964_del 2023-03-09 01:55:36,827 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030927a1a49fb1f04db4bb45b3beaecf8dc6 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309b7ed83e86ebb480fbb10faee5ca824c1 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309beb0560b7a674037913ac15ba3bb037f 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230309fec823224a2f4e7abc862491640daf44 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303090c6d7a77dbdd4150a39556745b75e1db_del 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309aa9b7d5917264d269dc6eeff4e21a1c5 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309bb4349f7ac904ce084506bf4674e0c82 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309e0f7092228534ccabbe807808edc1f89 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230309ee66da52298848a4a316e031c7adb822 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303090f8e236dd2b44d0aa74f234ad0abaf0b 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309251d64456774424d9d2445a93d454feb 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303093df924d792694c629ae9256c3bc125ef 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309584d72e480fb46e89925dd5811a9e8e7_del 2023-03-09 01:55:36,828 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309b28a61ad448d408b804acac511dc7db0 2023-03-09 01:55:37,010 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:37,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:37,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-09 01:55:37,011 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-09 01:55:37,233 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-09 01:55:37,233 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1678326923611-1678326936392 2023-03-09 01:55:37,233 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678326923611-1678326936392 2023-03-09 01:55:37,512 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:37,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:37,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-09 01:55:37,513 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-09 01:55:37,665 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678326923611-1678326936392 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/hbase-failsafe-snaptb-1678326923611-1678326936392 2023-03-09 01:55:38,480 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1678326923611-1678326936392 of table testCompactionWithHFileLink completed 2023-03-09 01:55:38,480 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678326923611-1678326936392 2023-03-09 01:55:38,480 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1678326923611-1678326936392 2023-03-09 01:55:38,480 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1678326923611-1678326936392 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1678326936407Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1678326936813 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1678326923611-1678326936392 at 1678326937233 (+420 ms)Loading Region manifests for hbase-failsafe-snaptb-1678326923611-1678326936392 at 1678326937233Writing data manifest for hbase-failsafe-snaptb-1678326923611-1678326936392 at 1678326937238 (+5 ms)Verifying snapshot: hbase-failsafe-snaptb-1678326923611-1678326936392 at 1678326937644 (+406 ms)Snapshot hbase-failsafe-snaptb-1678326923611-1678326936392 of table testCompactionWithHFileLink completed at 1678326938480 (+836 ms) 2023-03-09 01:55:38,481 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-09 01:55:38,483 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0760 sec 2023-03-09 01:55:38,513 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-09 01:55:38,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-09 01:55:38,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1678326923611-1678326936392 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-09 01:55:38,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678326923611" table: "testCompactionWithHFileLink" creation_time: 1678326923621 type: FLUSH version: 2 ttl: 0 ) 2023-03-09 01:55:38,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1678326923611 as table=testCompactionWithHFileLink 2023-03-09 01:55:38,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-09 01:55:38,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-09 01:55:38,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-09 01:55:38,963 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-09 01:55:38,964 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-09 01:55:38,964 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-09 01:55:38,965 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-09 01:55:38,971 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1678326923611" table: "testCompactionWithHFileLink" creation_time: 1678326923621 type: FLUSH version: 2 ttl: 0 2023-03-09 01:55:38,972 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink 2023-03-09 01:55:38,977 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-09 01:55:38,981 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309c026565af9bd49bd801e226238db7596 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230309c026565af9bd49bd801e226238db7596 2023-03-09 01:55:38,982 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309a51a7c9ebf2b48a4962233b0b0c2b438 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309a51a7c9ebf2b48a4962233b0b0c2b438 2023-03-09 01:55:38,983 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303093de2db0a7c6b4ec6b7e7280f8aec9b5b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303093de2db0a7c6b4ec6b7e7280f8aec9b5b 2023-03-09 01:55:38,983 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:38,987 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:38,990 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:38,993 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:38,996 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:38,999 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:39,002 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:39,005 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:39,009 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:39,012 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:39,015 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:39,017 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412 of snapshot=snaptb-1678326923611 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-09 01:55:39,022 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030924ba2d9bd7bb45c4b1ceda80e2874964_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023030924ba2d9bd7bb45c4b1ceda80e2874964_del 2023-03-09 01:55:39,023 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309584d72e480fb46e89925dd5811a9e8e7_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230309584d72e480fb46e89925dd5811a9e8e7_del 2023-03-09 01:55:39,024 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303090c6d7a77dbdd4150a39556745b75e1db_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303090c6d7a77dbdd4150a39556745b75e1db_del 2023-03-09 01:55:39,024 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,024 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,024 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,028 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/b764918ae73241b3bdbd646f18e6e4a2_SeqId_29_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/b764918ae73241b3bdbd646f18e6e4a2_SeqId_29_ 2023-03-09 01:55:39,028 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/446e175e02de45f09f63b31077a470c2_SeqId_29_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/446e175e02de45f09f63b31077a470c2_SeqId_29_ 2023-03-09 01:55:39,028 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/7bbcac145c534ba8bc25c71081434654 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/7bbcac145c534ba8bc25c71081434654 2023-03-09 01:55:39,029 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/080b00be8f2745859f51a21f5974bc6e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/080b00be8f2745859f51a21f5974bc6e 2023-03-09 01:55:39,029 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/2c95bc2ca9ad4805b61f6e90f117bcb8_SeqId_29_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/2c95bc2ca9ad4805b61f6e90f117bcb8_SeqId_29_ 2023-03-09 01:55:39,029 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 554613f5dee54006ad0ca4cc98415b67 of snapshot=snaptb-1678326923611 to region=abbc0d228ef2735615e5e8f7e1c367fc table=testCompactionWithHFileLink 2023-03-09 01:55:39,029 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6b9df6dda32a470bb7beedc493b4d7c2 of snapshot=snaptb-1678326923611 to region=18e9a9ff9da4b5322903ba24ba13b2e1 table=testCompactionWithHFileLink 2023-03-09 01:55:39,029 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/fceb5fa8161f4343a26f6d83b908baeb to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/fceb5fa8161f4343a26f6d83b908baeb 2023-03-09 01:55:39,030 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bd7eaca3d0bb49c881093d8bffbe4a61 of snapshot=snaptb-1678326923611 to region=00f768e6a69add6056570cdcb3ad746e table=testCompactionWithHFileLink 2023-03-09 01:55:39,035 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a19058eff3f84d83a01b0191d4a9958d of snapshot=snaptb-1678326923611 to region=abbc0d228ef2735615e5e8f7e1c367fc table=testCompactionWithHFileLink 2023-03-09 01:55:39,035 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9f28ae193d6b4e34bbb73a959c56c0f2 of snapshot=snaptb-1678326923611 to region=00f768e6a69add6056570cdcb3ad746e table=testCompactionWithHFileLink 2023-03-09 01:55:39,035 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 46f048b3bec44725bbd2a2dd1547ad65 of snapshot=snaptb-1678326923611 to region=18e9a9ff9da4b5322903ba24ba13b2e1 table=testCompactionWithHFileLink 2023-03-09 01:55:39,038 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c03f67bd5eda426d98fe2d2f23cddf3c of snapshot=snaptb-1678326923611 to region=abbc0d228ef2735615e5e8f7e1c367fc table=testCompactionWithHFileLink 2023-03-09 01:55:39,039 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e52084fb3ce043d3b3193b82c7c8c28d of snapshot=snaptb-1678326923611 to region=18e9a9ff9da4b5322903ba24ba13b2e1 table=testCompactionWithHFileLink 2023-03-09 01:55:39,039 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 82dcda8270744c87b9890f8a8573eb1a of snapshot=snaptb-1678326923611 to region=00f768e6a69add6056570cdcb3ad746e table=testCompactionWithHFileLink 2023-03-09 01:55:39,042 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c8a9da0c9dea4201985e0dda7f99ce78 of snapshot=snaptb-1678326923611 to region=00f768e6a69add6056570cdcb3ad746e table=testCompactionWithHFileLink 2023-03-09 01:55:39,042 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c092b1b2898b430cb5a6fd6ef235b622 of snapshot=snaptb-1678326923611 to region=18e9a9ff9da4b5322903ba24ba13b2e1 table=testCompactionWithHFileLink 2023-03-09 01:55:39,042 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9a43c7ed86e441a396bd23845a381994 of snapshot=snaptb-1678326923611 to region=abbc0d228ef2735615e5e8f7e1c367fc table=testCompactionWithHFileLink 2023-03-09 01:55:39,050 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/a86db3fc8d8c42e69269f5c7f5c31c00 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/a86db3fc8d8c42e69269f5c7f5c31c00 2023-03-09 01:55:39,050 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/db13546a2a52419da91ada0616887f46 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/db13546a2a52419da91ada0616887f46 2023-03-09 01:55:39,050 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 51a937efd2f644babe8351499c9fa15a of snapshot=snaptb-1678326923611 to region=00f768e6a69add6056570cdcb3ad746e table=testCompactionWithHFileLink 2023-03-09 01:55:39,050 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/572144be737042b4b014d1666d23d354 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/572144be737042b4b014d1666d23d354 2023-03-09 01:55:39,050 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink def1134fc58a4a269289ed9ac51b532e of snapshot=snaptb-1678326923611 to region=abbc0d228ef2735615e5e8f7e1c367fc table=testCompactionWithHFileLink 2023-03-09 01:55:39,051 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 60bbe8792b3d43189452c94ff064687b of snapshot=snaptb-1678326923611 to region=18e9a9ff9da4b5322903ba24ba13b2e1 table=testCompactionWithHFileLink 2023-03-09 01:55:39,055 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 647e37787cd149a190171dfdcdb08187 of snapshot=snaptb-1678326923611 to region=18e9a9ff9da4b5322903ba24ba13b2e1 table=testCompactionWithHFileLink 2023-03-09 01:55:39,056 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b54f40cf492f48f2a959cbed0a4fd66d of snapshot=snaptb-1678326923611 to region=00f768e6a69add6056570cdcb3ad746e table=testCompactionWithHFileLink 2023-03-09 01:55:39,057 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e23c8b4c586a48068d85ad2125091e46 of snapshot=snaptb-1678326923611 to region=abbc0d228ef2735615e5e8f7e1c367fc table=testCompactionWithHFileLink 2023-03-09 01:55:39,060 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e8e622fe17cc4feaaa821052b05b2f1d of snapshot=snaptb-1678326923611 to region=18e9a9ff9da4b5322903ba24ba13b2e1 table=testCompactionWithHFileLink 2023-03-09 01:55:39,060 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8c204b7188634b12a4b18c8cb9e96815 of snapshot=snaptb-1678326923611 to region=00f768e6a69add6056570cdcb3ad746e table=testCompactionWithHFileLink 2023-03-09 01:55:39,062 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f626b340147b41f481b99eceb82737c3 of snapshot=snaptb-1678326923611 to region=abbc0d228ef2735615e5e8f7e1c367fc table=testCompactionWithHFileLink 2023-03-09 01:55:39,064 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0a6c637662964e48a5d1d38c97d82cef of snapshot=snaptb-1678326923611 to region=18e9a9ff9da4b5322903ba24ba13b2e1 table=testCompactionWithHFileLink 2023-03-09 01:55:39,064 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6fd2b3985d2b4c7b812d559361310f38 of snapshot=snaptb-1678326923611 to region=00f768e6a69add6056570cdcb3ad746e table=testCompactionWithHFileLink 2023-03-09 01:55:39,065 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5c406707c2a64cd4ab677f3124d6af50 of snapshot=snaptb-1678326923611 to region=abbc0d228ef2735615e5e8f7e1c367fc table=testCompactionWithHFileLink 2023-03-09 01:55:39,069 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1678326923611" table: "testCompactionWithHFileLink" creation_time: 1678326923621 type: FLUSH version: 2 ttl: 0 2023-03-09 01:55:39,070 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678326939070"}]},"ts":"9223372036854775807"} 2023-03-09 01:55:39,071 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678326939070"}]},"ts":"9223372036854775807"} 2023-03-09 01:55:39,071 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1678326939070"}]},"ts":"9223372036854775807"} 2023-03-09 01:55:39,073 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-09 01:55:39,073 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 00f768e6a69add6056570cdcb3ad746e, NAME => 'testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 18e9a9ff9da4b5322903ba24ba13b2e1, NAME => 'testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => abbc0d228ef2735615e5e8f7e1c367fc, NAME => 'testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.', STARTKEY => '2', ENDKEY => ''}] 2023-03-09 01:55:39,073 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326939071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326939071"}]},"ts":"1678326939071"} 2023-03-09 01:55:39,073 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326939071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326939071"}]},"ts":"1678326939071"} 2023-03-09 01:55:39,073 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326939071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326939071"}]},"ts":"1678326939071"} 2023-03-09 01:55:39,074 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-09 01:55:39,074 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-09 01:55:39,075 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 00f768e6a69add6056570cdcb3ad746e, NAME => 'testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 18e9a9ff9da4b5322903ba24ba13b2e1, NAME => 'testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => abbc0d228ef2735615e5e8f7e1c367fc, NAME => 'testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.', STARTKEY => '2', ENDKEY => ''}] 2023-03-09 01:55:39,075 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1678326923611 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-09 01:55:39,077 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1678326923611" table: "testCompactionWithHFileLink" creation_time: 1678326923621 type: FLUSH version: 2 ttl: 0 ) in 532 msec 2023-03-09 01:55:39,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-09 01:55:39,159 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-09 01:55:39,159 INFO [Listener at localhost/33973] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1678326923611-1678326936392 2023-03-09 01:55:39,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1678326923611-1678326936392" 2023-03-09 01:55:39,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1678326923611-1678326936392 2023-03-09 01:55:39,168 INFO [Listener at localhost/33973] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-09 01:55:39,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-09 01:55:39,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-09 01:55:39,180 INFO [PEWorker-1] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-09 01:55:39,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-09 01:55:39,181 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326939180"}]},"ts":"1678326939180"} 2023-03-09 01:55:39,182 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-09 01:55:39,184 DEBUG [PEWorker-1] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-09 01:55:39,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, ASSIGN}] 2023-03-09 01:55:39,185 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, ASSIGN 2023-03-09 01:55:39,186 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, ASSIGN 2023-03-09 01:55:39,186 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, ASSIGN 2023-03-09 01:55:39,186 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-09 01:55:39,186 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-09 01:55:39,186 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-09 01:55:39,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-09 01:55:39,338 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=abbc0d228ef2735615e5e8f7e1c367fc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:39,338 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=00f768e6a69add6056570cdcb3ad746e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:39,338 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=18e9a9ff9da4b5322903ba24ba13b2e1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:39,338 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326939338"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326939338"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326939338"}]},"ts":"1678326939338"} 2023-03-09 01:55:39,338 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326939338"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326939338"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326939338"}]},"ts":"1678326939338"} 2023-03-09 01:55:39,338 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326939338"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326939338"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326939338"}]},"ts":"1678326939338"} 2023-03-09 01:55:39,340 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure abbc0d228ef2735615e5e8f7e1c367fc, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:39,340 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 00f768e6a69add6056570cdcb3ad746e, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:39,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure 18e9a9ff9da4b5322903ba24ba13b2e1, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:39,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-09 01:55:39,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:39,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 18e9a9ff9da4b5322903ba24ba13b2e1, NAME => 'testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.', STARTKEY => '1', ENDKEY => '2'} 2023-03-09 01:55:39,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:39,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,496 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,496 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1 2023-03-09 01:55:39,496 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1 2023-03-09 01:55:39,497 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18e9a9ff9da4b5322903ba24ba13b2e1 columnFamilyName family1 2023-03-09 01:55:39,512 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-46f048b3bec44725bbd2a2dd1547ad65 2023-03-09 01:55:39,517 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-6b9df6dda32a470bb7beedc493b4d7c2 2023-03-09 01:55:39,522 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-c092b1b2898b430cb5a6fd6ef235b622 2023-03-09 01:55:39,527 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e52084fb3ce043d3b3193b82c7c8c28d 2023-03-09 01:55:39,527 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(310): Store=18e9a9ff9da4b5322903ba24ba13b2e1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:39,528 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,528 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2 2023-03-09 01:55:39,528 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2 2023-03-09 01:55:39,529 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18e9a9ff9da4b5322903ba24ba13b2e1 columnFamilyName family2 2023-03-09 01:55:39,536 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-0a6c637662964e48a5d1d38c97d82cef 2023-03-09 01:55:39,542 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-60bbe8792b3d43189452c94ff064687b 2023-03-09 01:55:39,547 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-647e37787cd149a190171dfdcdb08187 2023-03-09 01:55:39,552 DEBUG [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e8e622fe17cc4feaaa821052b05b2f1d 2023-03-09 01:55:39,553 INFO [StoreOpener-18e9a9ff9da4b5322903ba24ba13b2e1-1] regionserver.HStore(310): Store=18e9a9ff9da4b5322903ba24ba13b2e1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:39,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:39,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:55:39,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 18e9a9ff9da4b5322903ba24ba13b2e1; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59272417, jitterRate=-0.11677215993404388}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:39,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:39,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1., pid=103, masterSystemTime=1678326939491 2023-03-09 01:55:39,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:39,560 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-09 01:55:39,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:39,560 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-09 01:55:39,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:39,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:39,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => abbc0d228ef2735615e5e8f7e1c367fc, NAME => 'testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.', STARTKEY => '2', ENDKEY => ''} 2023-03-09 01:55:39,563 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=18e9a9ff9da4b5322903ba24ba13b2e1, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:39,563 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326939563"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326939563"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326939563"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326939563"}]},"ts":"1678326939563"} 2023-03-09 01:55:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,565 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,566 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-09 01:55:39,566 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HStore(1912): 18e9a9ff9da4b5322903ba24ba13b2e1/family2 is initiating minor compaction (all files) 2023-03-09 01:55:39,566 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 18e9a9ff9da4b5322903ba24ba13b2e1/family2 in testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:39,566 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-60bbe8792b3d43189452c94ff064687b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-647e37787cd149a190171dfdcdb08187, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e8e622fe17cc4feaaa821052b05b2f1d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-0a6c637662964e48a5d1d38c97d82cef] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp, totalSize=20.4 K 2023-03-09 01:55:39,567 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-03-09 01:55:39,567 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure 18e9a9ff9da4b5322903ba24ba13b2e1, server=jenkins-hbase4.apache.org,46511,1678326819757 in 224 msec 2023-03-09 01:55:39,567 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1 2023-03-09 01:55:39,567 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1 2023-03-09 01:55:39,567 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-09 01:55:39,567 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1912): 18e9a9ff9da4b5322903ba24ba13b2e1/family1 is initiating minor compaction (all files) 2023-03-09 01:55:39,567 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 18e9a9ff9da4b5322903ba24ba13b2e1/family1 in testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:55:39,567 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region abbc0d228ef2735615e5e8f7e1c367fc columnFamilyName family1 2023-03-09 01:55:39,567 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-6b9df6dda32a470bb7beedc493b4d7c2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-46f048b3bec44725bbd2a2dd1547ad65, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e52084fb3ce043d3b3193b82c7c8c28d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-c092b1b2898b430cb5a6fd6ef235b622] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp, totalSize=21.6 K 2023-03-09 01:55:39,568 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-60bbe8792b3d43189452c94ff064687b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916809 2023-03-09 01:55:39,568 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=18e9a9ff9da4b5322903ba24ba13b2e1, ASSIGN in 383 msec 2023-03-09 01:55:39,569 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-6b9df6dda32a470bb7beedc493b4d7c2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916809 2023-03-09 01:55:39,569 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-647e37787cd149a190171dfdcdb08187, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:39,569 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-46f048b3bec44725bbd2a2dd1547ad65, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:39,569 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e8e622fe17cc4feaaa821052b05b2f1d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:39,570 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e52084fb3ce043d3b3193b82c7c8c28d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:39,570 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-0a6c637662964e48a5d1d38c97d82cef, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921801 2023-03-09 01:55:39,572 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-c092b1b2898b430cb5a6fd6ef235b622, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921801 2023-03-09 01:55:39,582 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-554613f5dee54006ad0ca4cc98415b67 2023-03-09 01:55:39,592 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-9a43c7ed86e441a396bd23845a381994 2023-03-09 01:55:39,608 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-a19058eff3f84d83a01b0191d4a9958d 2023-03-09 01:55:39,614 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] throttle.PressureAwareThroughputController(145): 18e9a9ff9da4b5322903ba24ba13b2e1#family2#compaction#113 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:55:39,616 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 18e9a9ff9da4b5322903ba24ba13b2e1#family1#compaction#114 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:55:39,621 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-c03f67bd5eda426d98fe2d2f23cddf3c 2023-03-09 01:55:39,621 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(310): Store=abbc0d228ef2735615e5e8f7e1c367fc/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:39,621 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,622 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2 2023-03-09 01:55:39,622 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2 2023-03-09 01:55:39,622 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region abbc0d228ef2735615e5e8f7e1c367fc columnFamilyName family2 2023-03-09 01:55:39,635 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-5c406707c2a64cd4ab677f3124d6af50 2023-03-09 01:55:39,640 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-def1134fc58a4a269289ed9ac51b532e 2023-03-09 01:55:39,644 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family2/bed436ffc8634b7baab14ff6073422ad as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/bed436ffc8634b7baab14ff6073422ad 2023-03-09 01:55:39,649 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-e23c8b4c586a48068d85ad2125091e46 2023-03-09 01:55:39,650 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 18e9a9ff9da4b5322903ba24ba13b2e1/family2 of 18e9a9ff9da4b5322903ba24ba13b2e1 into bed436ffc8634b7baab14ff6073422ad(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:55:39,650 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:39,650 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1., storeName=18e9a9ff9da4b5322903ba24ba13b2e1/family2, priority=12, startTime=1678326939560; duration=0sec 2023-03-09 01:55:39,650 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:39,655 DEBUG [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-f626b340147b41f481b99eceb82737c3 2023-03-09 01:55:39,655 INFO [StoreOpener-abbc0d228ef2735615e5e8f7e1c367fc-1] regionserver.HStore(310): Store=abbc0d228ef2735615e5e8f7e1c367fc/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:39,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:39,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:55:39,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened abbc0d228ef2735615e5e8f7e1c367fc; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73759740, jitterRate=0.09910577535629272}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:39,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:39,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc., pid=101, masterSystemTime=1678326939491 2023-03-09 01:55:39,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:39,662 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-09 01:55:39,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-09 01:55:39,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:39,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:39,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:39,664 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=abbc0d228ef2735615e5e8f7e1c367fc, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:39,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 00f768e6a69add6056570cdcb3ad746e, NAME => 'testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.', STARTKEY => '', ENDKEY => '1'} 2023-03-09 01:55:39,664 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326939664"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326939664"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326939664"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326939664"}]},"ts":"1678326939664"} 2023-03-09 01:55:39,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:39,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,665 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-09 01:55:39,666 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HStore(1912): abbc0d228ef2735615e5e8f7e1c367fc/family1 is initiating minor compaction (all files) 2023-03-09 01:55:39,666 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegion(2259): Starting compaction of abbc0d228ef2735615e5e8f7e1c367fc/family1 in testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:39,666 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-554613f5dee54006ad0ca4cc98415b67, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-a19058eff3f84d83a01b0191d4a9958d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-c03f67bd5eda426d98fe2d2f23cddf3c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-9a43c7ed86e441a396bd23845a381994] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp, totalSize=21.6 K 2023-03-09 01:55:39,667 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-03-09 01:55:39,667 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,667 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure abbc0d228ef2735615e5e8f7e1c367fc, server=jenkins-hbase4.apache.org,46511,1678326819757 in 326 msec 2023-03-09 01:55:39,668 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-554613f5dee54006ad0ca4cc98415b67, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916810 2023-03-09 01:55:39,668 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-a19058eff3f84d83a01b0191d4a9958d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:39,668 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=abbc0d228ef2735615e5e8f7e1c367fc, ASSIGN in 483 msec 2023-03-09 01:55:39,668 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1 2023-03-09 01:55:39,669 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1 2023-03-09 01:55:39,669 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-c03f67bd5eda426d98fe2d2f23cddf3c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:39,669 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 00f768e6a69add6056570cdcb3ad746e columnFamilyName family1 2023-03-09 01:55:39,669 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-9a43c7ed86e441a396bd23845a381994, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921802 2023-03-09 01:55:39,677 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-82dcda8270744c87b9890f8a8573eb1a 2023-03-09 01:55:39,686 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-9f28ae193d6b4e34bbb73a959c56c0f2 2023-03-09 01:55:39,692 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] throttle.PressureAwareThroughputController(145): abbc0d228ef2735615e5e8f7e1c367fc#family1#compaction#115 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:55:39,701 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-bd7eaca3d0bb49c881093d8bffbe4a61 2023-03-09 01:55:39,713 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-c8a9da0c9dea4201985e0dda7f99ce78 2023-03-09 01:55:39,714 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(310): Store=00f768e6a69add6056570cdcb3ad746e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:39,714 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,715 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2 2023-03-09 01:55:39,715 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2 2023-03-09 01:55:39,715 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 00f768e6a69add6056570cdcb3ad746e columnFamilyName family2 2023-03-09 01:55:39,721 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-51a937efd2f644babe8351499c9fa15a 2023-03-09 01:55:39,727 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-6fd2b3985d2b4c7b812d559361310f38 2023-03-09 01:55:39,732 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-8c204b7188634b12a4b18c8cb9e96815 2023-03-09 01:55:39,737 DEBUG [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(539): loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-b54f40cf492f48f2a959cbed0a4fd66d 2023-03-09 01:55:39,737 INFO [StoreOpener-00f768e6a69add6056570cdcb3ad746e-1] regionserver.HStore(310): Store=00f768e6a69add6056570cdcb3ad746e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:39,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:39,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:55:39,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 00f768e6a69add6056570cdcb3ad746e; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74823019, jitterRate=0.11494986712932587}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:39,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:39,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e., pid=102, masterSystemTime=1678326939491 2023-03-09 01:55:39,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-09 01:55:39,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-09 01:55:39,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:39,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:39,745 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=00f768e6a69add6056570cdcb3ad746e, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:39,745 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1678326939744"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326939744"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326939744"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326939744"}]},"ts":"1678326939744"} 2023-03-09 01:55:39,747 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-03-09 01:55:39,747 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 00f768e6a69add6056570cdcb3ad746e, server=jenkins-hbase4.apache.org,46511,1678326819757 in 406 msec 2023-03-09 01:55:39,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=98, resume processing ppid=97 2023-03-09 01:55:39,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=00f768e6a69add6056570cdcb3ad746e, ASSIGN in 563 msec 2023-03-09 01:55:39,754 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-60bbe8792b3d43189452c94ff064687b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-647e37787cd149a190171dfdcdb08187, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e8e622fe17cc4feaaa821052b05b2f1d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-0a6c637662964e48a5d1d38c97d82cef] to archive 2023-03-09 01:55:39,754 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326939754"}]},"ts":"1678326939754"} 2023-03-09 01:55:39,754 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:39,755 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-09 01:55:39,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-60bbe8792b3d43189452c94ff064687b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-60bbe8792b3d43189452c94ff064687b 2023-03-09 01:55:39,757 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-09 01:55:39,757 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-647e37787cd149a190171dfdcdb08187 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-647e37787cd149a190171dfdcdb08187 2023-03-09 01:55:39,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e8e622fe17cc4feaaa821052b05b2f1d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e8e622fe17cc4feaaa821052b05b2f1d 2023-03-09 01:55:39,758 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 584 msec 2023-03-09 01:55:39,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-0a6c637662964e48a5d1d38c97d82cef to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family2/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-0a6c637662964e48a5d1d38c97d82cef 2023-03-09 01:55:39,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-09 01:55:39,784 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-09 01:55:39,797 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:39,803 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:39,812 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:39,813 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:39,820 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-09 01:55:39,835 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-09 01:55:39,835 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:39,835 DEBUG [Listener at localhost/33973] zookeeper.ReadOnlyZKClient(139): Connect 0x683e65d4 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:39,840 DEBUG [Listener at localhost/33973] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6f8fe831, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:39,840 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:39,840 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:39,840 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:39,916 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030993619d22d80242378349a8fc26c9d556 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023030993619d22d80242378349a8fc26c9d556 2023-03-09 01:55:39,917 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:39,929 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:55:39,933 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47530, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:55:39,940 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:55:39,942 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41982, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:55:39,954 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309/family1/8b07eb6f000c4f9ab74f04a1ed205752 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-09 01:55:39,955 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309/family1/8b07eb6f000c4f9ab74f04a1ed205752}] 2023-03-09 01:55:39,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309/family1/8b07eb6f000c4f9ab74f04a1ed205752 for inclusion in abbc0d228ef2735615e5e8f7e1c367fc/family1 2023-03-09 01:55:39,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-09 01:55:39,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-09 01:55:39,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:39,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309/family1/8b07eb6f000c4f9ab74f04a1ed205752 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ghloguisjcn0cf1rcuvlm48a093pfsaniveek1lo183uv8oucsi202ivf3sutpfv/family1/8b07eb6f000c4f9ab74f04a1ed205752 2023-03-09 01:55:39,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ghloguisjcn0cf1rcuvlm48a093pfsaniveek1lo183uv8oucsi202ivf3sutpfv/family1/8b07eb6f000c4f9ab74f04a1ed205752 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/6796b1c46180438ea353d37b6f2557c1_SeqId_34_ 2023-03-09 01:55:39,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ghloguisjcn0cf1rcuvlm48a093pfsaniveek1lo183uv8oucsi202ivf3sutpfv/family1/8b07eb6f000c4f9ab74f04a1ed205752 into abbc0d228ef2735615e5e8f7e1c367fc/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/6796b1c46180438ea353d37b6f2557c1_SeqId_34_ - updating store file list. 2023-03-09 01:55:39,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/6796b1c46180438ea353d37b6f2557c1_SeqId_34_ into abbc0d228ef2735615e5e8f7e1c367fc/family1 2023-03-09 01:55:39,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ghloguisjcn0cf1rcuvlm48a093pfsaniveek1lo183uv8oucsi202ivf3sutpfv/family1/8b07eb6f000c4f9ab74f04a1ed205752 into abbc0d228ef2735615e5e8f7e1c367fc/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/6796b1c46180438ea353d37b6f2557c1_SeqId_34_) 2023-03-09 01:55:39,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__ghloguisjcn0cf1rcuvlm48a093pfsaniveek1lo183uv8oucsi202ivf3sutpfv/family1/8b07eb6f000c4f9ab74f04a1ed205752 2023-03-09 01:55:39,973 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:39,973 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:39,975 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023030926fbb9aa3aa44553b2c472dd9a8248aa 2023-03-09 01:55:39,976 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303092fff17db4b7a4b72aaa997641ec92178 2023-03-09 01:55:39,977 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303095027c41d596349ae906aa100b41dcb96 2023-03-09 01:55:39,978 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303097ae47ba608884ae8b9468cff72192763 2023-03-09 01:55:39,978 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-09 01:55:39,978 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230309' 2023-03-09 01:55:39,978 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:40,435 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/.tmp/family1/40883a31fd9947658b5995954c419be3 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/40883a31fd9947658b5995954c419be3 2023-03-09 01:55:40,440 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 18e9a9ff9da4b5322903ba24ba13b2e1/family1 of 18e9a9ff9da4b5322903ba24ba13b2e1 into 40883a31fd9947658b5995954c419be3(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:55:40,440 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:40,440 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1., storeName=18e9a9ff9da4b5322903ba24ba13b2e1/family1, priority=12, startTime=1678326939559; duration=0sec 2023-03-09 01:55:40,440 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-09 01:55:40,440 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-09 01:55:40,442 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-09 01:55:40,442 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1912): abbc0d228ef2735615e5e8f7e1c367fc/family2 is initiating minor compaction (all files) 2023-03-09 01:55:40,442 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of abbc0d228ef2735615e5e8f7e1c367fc/family2 in testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:55:40,442 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-def1134fc58a4a269289ed9ac51b532e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-e23c8b4c586a48068d85ad2125091e46, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-f626b340147b41f481b99eceb82737c3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-5c406707c2a64cd4ab677f3124d6af50] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp, totalSize=20.4 K 2023-03-09 01:55:40,443 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-def1134fc58a4a269289ed9ac51b532e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916810 2023-03-09 01:55:40,443 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-e23c8b4c586a48068d85ad2125091e46, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:40,444 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-f626b340147b41f481b99eceb82737c3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:40,444 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-5c406707c2a64cd4ab677f3124d6af50, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921802 2023-03-09 01:55:40,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-6b9df6dda32a470bb7beedc493b4d7c2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-46f048b3bec44725bbd2a2dd1547ad65, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e52084fb3ce043d3b3193b82c7c8c28d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-c092b1b2898b430cb5a6fd6ef235b622] to archive 2023-03-09 01:55:40,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:40,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-6b9df6dda32a470bb7beedc493b4d7c2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-6b9df6dda32a470bb7beedc493b4d7c2 2023-03-09 01:55:40,457 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-46f048b3bec44725bbd2a2dd1547ad65 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-46f048b3bec44725bbd2a2dd1547ad65 2023-03-09 01:55:40,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e52084fb3ce043d3b3193b82c7c8c28d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-e52084fb3ce043d3b3193b82c7c8c28d 2023-03-09 01:55:40,459 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-c092b1b2898b430cb5a6fd6ef235b622 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/testCompactionWithHFileLink=18e9a9ff9da4b5322903ba24ba13b2e1-c092b1b2898b430cb5a6fd6ef235b622 2023-03-09 01:55:40,461 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): abbc0d228ef2735615e5e8f7e1c367fc#family2#compaction#116 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:55:40,717 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309a428f43716184cb5a4f1e76aefd605af to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230309a428f43716184cb5a4f1e76aefd605af 2023-03-09 01:55:40,718 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303099dc21a529f014ffeac46c85645f8605c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303099dc21a529f014ffeac46c85645f8605c 2023-03-09 01:55:40,718 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:40,718 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:40,744 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309/family1/fefe9240a740442fb07edd016573285a first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-09 01:55:40,745 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309/family1/fefe9240a740442fb07edd016573285a}] 2023-03-09 01:55:40,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309/family1/fefe9240a740442fb07edd016573285a for inclusion in 18e9a9ff9da4b5322903ba24ba13b2e1/family1 2023-03-09 01:55:40,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-09 01:55:40,750 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309/family1/1000a7c0473b4850b1b4922fda7f39a7 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-09 01:55:40,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-09 01:55:40,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:55:40,751 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=32 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309/family1/1000a7c0473b4850b1b4922fda7f39a7}] 2023-03-09 01:55:40,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309/family1/fefe9240a740442fb07edd016573285a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__jh2biklcu7g1qrh9n50beesshcmkmdue1e3dmquj5t9v7bv0l7vtvrkac9sd2fic/family1/fefe9240a740442fb07edd016573285a 2023-03-09 01:55:40,754 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309/family1/1000a7c0473b4850b1b4922fda7f39a7 for inclusion in 00f768e6a69add6056570cdcb3ad746e/family1 2023-03-09 01:55:40,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__jh2biklcu7g1qrh9n50beesshcmkmdue1e3dmquj5t9v7bv0l7vtvrkac9sd2fic/family1/fefe9240a740442fb07edd016573285a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/25a9a8af294640afa2bf64a21cc3781b_SeqId_36_ 2023-03-09 01:55:40,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__jh2biklcu7g1qrh9n50beesshcmkmdue1e3dmquj5t9v7bv0l7vtvrkac9sd2fic/family1/fefe9240a740442fb07edd016573285a into 18e9a9ff9da4b5322903ba24ba13b2e1/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/25a9a8af294640afa2bf64a21cc3781b_SeqId_36_ - updating store file list. 2023-03-09 01:55:40,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-09 01:55:40,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last=1 2023-03-09 01:55:40,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:40,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309/family1/1000a7c0473b4850b1b4922fda7f39a7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__mer1ubsbgmcled4cd13bo63jtfc8o79upjhlgtho8s8as79g4bep29663tahrg2c/family1/1000a7c0473b4850b1b4922fda7f39a7 2023-03-09 01:55:40,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__mer1ubsbgmcled4cd13bo63jtfc8o79upjhlgtho8s8as79g4bep29663tahrg2c/family1/1000a7c0473b4850b1b4922fda7f39a7 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/367775e718c2450a927fa5a1a179205c_SeqId_34_ 2023-03-09 01:55:40,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/25a9a8af294640afa2bf64a21cc3781b_SeqId_36_ into 18e9a9ff9da4b5322903ba24ba13b2e1/family1 2023-03-09 01:55:40,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__jh2biklcu7g1qrh9n50beesshcmkmdue1e3dmquj5t9v7bv0l7vtvrkac9sd2fic/family1/fefe9240a740442fb07edd016573285a into 18e9a9ff9da4b5322903ba24ba13b2e1/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/family1/25a9a8af294640afa2bf64a21cc3781b_SeqId_36_) 2023-03-09 01:55:40,760 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__mer1ubsbgmcled4cd13bo63jtfc8o79upjhlgtho8s8as79g4bep29663tahrg2c/family1/1000a7c0473b4850b1b4922fda7f39a7 into 00f768e6a69add6056570cdcb3ad746e/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/367775e718c2450a927fa5a1a179205c_SeqId_34_ - updating store file list. 2023-03-09 01:55:40,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__jh2biklcu7g1qrh9n50beesshcmkmdue1e3dmquj5t9v7bv0l7vtvrkac9sd2fic/family1/fefe9240a740442fb07edd016573285a 2023-03-09 01:55:40,762 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:40,763 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:40,764 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/367775e718c2450a927fa5a1a179205c_SeqId_34_ into 00f768e6a69add6056570cdcb3ad746e/family1 2023-03-09 01:55:40,764 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__mer1ubsbgmcled4cd13bo63jtfc8o79upjhlgtho8s8as79g4bep29663tahrg2c/family1/1000a7c0473b4850b1b4922fda7f39a7 into 00f768e6a69add6056570cdcb3ad746e/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/367775e718c2450a927fa5a1a179205c_SeqId_34_) 2023-03-09 01:55:40,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testCompactionWithHFileLink__mer1ubsbgmcled4cd13bo63jtfc8o79upjhlgtho8s8as79g4bep29663tahrg2c/family1/1000a7c0473b4850b1b4922fda7f39a7 2023-03-09 01:55:40,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:40,765 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303091fe5f30f7e634dbe84a54aee8f8aa8c7 2023-03-09 01:55:40,766 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303097fad5ee327b74a0390f421128a1e3a84 2023-03-09 01:55:40,768 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303098f1d675d82984c9a82b77912a171135b 2023-03-09 01:55:40,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-09 01:55:40,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(1912): 00f768e6a69add6056570cdcb3ad746e/family1 is initiating minor compaction (all files) 2023-03-09 01:55:40,769 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230309c2125126c19a4144af37a71d1c82ea99 2023-03-09 01:55:40,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@208668da; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-09 01:55:40,769 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-09 01:55:40,769 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230309' 2023-03-09 01:55:40,770 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:40,771 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:40,772 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:40,773 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230309204f9b5aa0e6475fa24d0662019a7d8f 2023-03-09 01:55:40,775 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023030945c5fbfeedbf4d67a5b8d5bb2c6a34a5 2023-03-09 01:55:40,776 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303098862e949f298429ea4be0cd0a83a4914 2023-03-09 01:55:40,777 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230309999d30196311418a99c65cb7933cf412 2023-03-09 01:55:40,777 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-09 01:55:40,777 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230309' 2023-03-09 01:55:40,777 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230309 for table='testCompactionWithHFileLink' and column='family1' 2023-03-09 01:55:40,778 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-09 01:55:40,780 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:40,785 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:40,789 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:40,791 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:40,798 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:40,799 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:40,817 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=281 (was 250) - Thread LEAK? -, OpenFileDescriptor=745 (was 626) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=142 (was 152), ProcessCount=171 (was 171), AvailableMemoryMB=7951 (was 8083) 2023-03-09 01:55:40,830 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=281, OpenFileDescriptor=745, MaxFileDescriptor=60000, SystemLoadAverage=142, ProcessCount=171, AvailableMemoryMB=7950 2023-03-09 01:55:40,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-09 01:55:40,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-09 01:55:40,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-09 01:55:40,846 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-09 01:55:40,848 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 9 msec 2023-03-09 01:55:40,924 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family1/a84aceeff8044fd9a132eb9d8e55b002 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/a84aceeff8044fd9a132eb9d8e55b002 2023-03-09 01:55:40,929 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in abbc0d228ef2735615e5e8f7e1c367fc/family1 of abbc0d228ef2735615e5e8f7e1c367fc into a84aceeff8044fd9a132eb9d8e55b002(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:40,929 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:40,929 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc., storeName=abbc0d228ef2735615e5e8f7e1c367fc/family1, priority=12, startTime=1678326939661; duration=1sec 2023-03-09 01:55:40,929 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-09 01:55:40,929 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 00f768e6a69add6056570cdcb3ad746e/family1 in testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:40,929 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-bd7eaca3d0bb49c881093d8bffbe4a61, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-9f28ae193d6b4e34bbb73a959c56c0f2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-82dcda8270744c87b9890f8a8573eb1a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-c8a9da0c9dea4201985e0dda7f99ce78, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/367775e718c2450a927fa5a1a179205c_SeqId_34_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp, totalSize=28.9 K 2023-03-09 01:55:40,930 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-bd7eaca3d0bb49c881093d8bffbe4a61, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916808 2023-03-09 01:55:40,930 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-9f28ae193d6b4e34bbb73a959c56c0f2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:40,931 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-82dcda8270744c87b9890f8a8573eb1a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:40,931 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-c8a9da0c9dea4201985e0dda7f99ce78, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921801 2023-03-09 01:55:40,932 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.Compactor(207): Compacting 367775e718c2450a927fa5a1a179205c_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1678326916808 2023-03-09 01:55:40,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-09 01:55:40,945 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] throttle.PressureAwareThroughputController(145): 00f768e6a69add6056570cdcb3ad746e#family1#compaction#117 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-03-09 01:55:40,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-09 01:55:40,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-09 01:55:40,948 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:55:40,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-09 01:55:40,948 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:55:40,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-09 01:55:40,950 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:40,950 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:40,950 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:40,951 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953 empty. 2023-03-09 01:55:40,951 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73 empty. 2023-03-09 01:55:40,951 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362 empty. 2023-03-09 01:55:40,951 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:40,951 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:40,951 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:40,951 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-09 01:55:40,960 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-554613f5dee54006ad0ca4cc98415b67, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-a19058eff3f84d83a01b0191d4a9958d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-c03f67bd5eda426d98fe2d2f23cddf3c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-9a43c7ed86e441a396bd23845a381994] to archive 2023-03-09 01:55:40,961 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:40,962 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-554613f5dee54006ad0ca4cc98415b67 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-554613f5dee54006ad0ca4cc98415b67 2023-03-09 01:55:40,963 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-a19058eff3f84d83a01b0191d4a9958d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-a19058eff3f84d83a01b0191d4a9958d 2023-03-09 01:55:40,964 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-c03f67bd5eda426d98fe2d2f23cddf3c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-c03f67bd5eda426d98fe2d2f23cddf3c 2023-03-09 01:55:40,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-9a43c7ed86e441a396bd23845a381994 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family1/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-9a43c7ed86e441a396bd23845a381994 2023-03-09 01:55:41,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-09 01:55:41,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-09 01:55:41,362 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-09 01:55:41,364 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => b0d3ba46f6423b83104f4f0f94be0362, NAME => 'ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:55:41,364 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 11ddcd7bf8df7d1e6665f6c9cc6acd73, NAME => 'ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:55:41,364 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => fd53913779b3fcab65c638e819ce5953, NAME => 'ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:55:41,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-09 01:55:41,678 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/.tmp/family2/d6892f987be94bb2807c09a88875a530 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/d6892f987be94bb2807c09a88875a530 2023-03-09 01:55:41,682 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in abbc0d228ef2735615e5e8f7e1c367fc/family2 of abbc0d228ef2735615e5e8f7e1c367fc into d6892f987be94bb2807c09a88875a530(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:41,683 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:55:41,683 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc., storeName=abbc0d228ef2735615e5e8f7e1c367fc/family2, priority=12, startTime=1678326939662; duration=1sec 2023-03-09 01:55:41,683 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-09 01:55:41,683 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-09 01:55:41,685 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-09 01:55:41,685 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1912): 00f768e6a69add6056570cdcb3ad746e/family2 is initiating minor compaction (all files) 2023-03-09 01:55:41,685 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 00f768e6a69add6056570cdcb3ad746e/family2 in testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:55:41,685 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-51a937efd2f644babe8351499c9fa15a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-b54f40cf492f48f2a959cbed0a4fd66d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-8c204b7188634b12a4b18c8cb9e96815, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-6fd2b3985d2b4c7b812d559361310f38] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp, totalSize=20.4 K 2023-03-09 01:55:41,686 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-51a937efd2f644babe8351499c9fa15a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326916808 2023-03-09 01:55:41,686 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-b54f40cf492f48f2a959cbed0a4fd66d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326918614 2023-03-09 01:55:41,687 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-8c204b7188634b12a4b18c8cb9e96815, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326920401 2023-03-09 01:55:41,687 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-6fd2b3985d2b4c7b812d559361310f38, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326921801 2023-03-09 01:55:41,699 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): 00f768e6a69add6056570cdcb3ad746e#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-03-09 01:55:41,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-def1134fc58a4a269289ed9ac51b532e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-e23c8b4c586a48068d85ad2125091e46, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-f626b340147b41f481b99eceb82737c3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-5c406707c2a64cd4ab677f3124d6af50] to archive 2023-03-09 01:55:41,757 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:41,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-def1134fc58a4a269289ed9ac51b532e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-def1134fc58a4a269289ed9ac51b532e 2023-03-09 01:55:41,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-e23c8b4c586a48068d85ad2125091e46 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-e23c8b4c586a48068d85ad2125091e46 2023-03-09 01:55:41,761 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-f626b340147b41f481b99eceb82737c3 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-f626b340147b41f481b99eceb82737c3 2023-03-09 01:55:41,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-5c406707c2a64cd4ab677f3124d6af50 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/family2/testCompactionWithHFileLink=abbc0d228ef2735615e5e8f7e1c367fc-5c406707c2a64cd4ab677f3124d6af50 2023-03-09 01:55:41,765 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family1/80dfcf4fa58b4422bdfa3d30f822cb48 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/80dfcf4fa58b4422bdfa3d30f822cb48 2023-03-09 01:55:41,769 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 00f768e6a69add6056570cdcb3ad746e/family1 of 00f768e6a69add6056570cdcb3ad746e into 80dfcf4fa58b4422bdfa3d30f822cb48(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:55:41,769 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:41,769 INFO [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1678326940769; duration=0sec 2023-03-09 01:55:41,769 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-09 01:55:41,769 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-09 01:55:41,769 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing fd53913779b3fcab65c638e819ce5953, disabling compactions & flushes 2023-03-09 01:55:41,770 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. after waiting 0 ms 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:41,770 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-09 01:55:41,770 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:41,770 DEBUG [RS:0;jenkins-hbase4:46511-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. because compaction request was cancelled 2023-03-09 01:55:41,770 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 11ddcd7bf8df7d1e6665f6c9cc6acd73, disabling compactions & flushes 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:41,770 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing b0d3ba46f6423b83104f4f0f94be0362, disabling compactions & flushes 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:41,770 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. after waiting 0 ms 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. after waiting 0 ms 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:41,770 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:41,771 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:41,771 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:41,771 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:41,771 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:41,772 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:55:41,773 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678326941773"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326941773"}]},"ts":"1678326941773"} 2023-03-09 01:55:41,773 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678326941773"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326941773"}]},"ts":"1678326941773"} 2023-03-09 01:55:41,773 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678326941773"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326941773"}]},"ts":"1678326941773"} 2023-03-09 01:55:41,775 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-09 01:55:41,776 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:55:41,776 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326941776"}]},"ts":"1678326941776"} 2023-03-09 01:55:41,777 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-09 01:55:41,780 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=fd53913779b3fcab65c638e819ce5953, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=11ddcd7bf8df7d1e6665f6c9cc6acd73, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b0d3ba46f6423b83104f4f0f94be0362, ASSIGN}] 2023-03-09 01:55:41,782 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b0d3ba46f6423b83104f4f0f94be0362, ASSIGN 2023-03-09 01:55:41,782 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=11ddcd7bf8df7d1e6665f6c9cc6acd73, ASSIGN 2023-03-09 01:55:41,782 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=fd53913779b3fcab65c638e819ce5953, ASSIGN 2023-03-09 01:55:41,782 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b0d3ba46f6423b83104f4f0f94be0362, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:55:41,782 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=fd53913779b3fcab65c638e819ce5953, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:55:41,782 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=11ddcd7bf8df7d1e6665f6c9cc6acd73, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:55:41,795 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:55:41,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-bd7eaca3d0bb49c881093d8bffbe4a61, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-9f28ae193d6b4e34bbb73a959c56c0f2, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-82dcda8270744c87b9890f8a8573eb1a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-c8a9da0c9dea4201985e0dda7f99ce78, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/367775e718c2450a927fa5a1a179205c_SeqId_34_] to archive 2023-03-09 01:55:41,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:41,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-bd7eaca3d0bb49c881093d8bffbe4a61 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-bd7eaca3d0bb49c881093d8bffbe4a61 2023-03-09 01:55:41,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-9f28ae193d6b4e34bbb73a959c56c0f2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-9f28ae193d6b4e34bbb73a959c56c0f2 2023-03-09 01:55:41,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-82dcda8270744c87b9890f8a8573eb1a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-82dcda8270744c87b9890f8a8573eb1a 2023-03-09 01:55:41,861 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-c8a9da0c9dea4201985e0dda7f99ce78 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-c8a9da0c9dea4201985e0dda7f99ce78 2023-03-09 01:55:41,862 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/367775e718c2450a927fa5a1a179205c_SeqId_34_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family1/367775e718c2450a927fa5a1a179205c_SeqId_34_ 2023-03-09 01:55:41,935 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=fd53913779b3fcab65c638e819ce5953, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:41,935 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=11ddcd7bf8df7d1e6665f6c9cc6acd73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:41,935 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=b0d3ba46f6423b83104f4f0f94be0362, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:41,935 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678326941935"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326941935"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326941935"}]},"ts":"1678326941935"} 2023-03-09 01:55:41,935 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678326941935"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326941935"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326941935"}]},"ts":"1678326941935"} 2023-03-09 01:55:41,935 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678326941935"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326941935"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326941935"}]},"ts":"1678326941935"} 2023-03-09 01:55:41,937 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=106, state=RUNNABLE; OpenRegionProcedure fd53913779b3fcab65c638e819ce5953, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:41,937 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure 11ddcd7bf8df7d1e6665f6c9cc6acd73, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:41,937 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure b0d3ba46f6423b83104f4f0f94be0362, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:55:42,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-09 01:55:42,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:42,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 11ddcd7bf8df7d1e6665f6c9cc6acd73, NAME => 'ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73.', STARTKEY => '1', ENDKEY => '2'} 2023-03-09 01:55:42,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:42,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:42,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:42,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:42,092 INFO [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:42,094 DEBUG [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 2023-03-09 01:55:42,094 DEBUG [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 2023-03-09 01:55:42,095 INFO [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11ddcd7bf8df7d1e6665f6c9cc6acd73 columnFamilyName family1 2023-03-09 01:55:42,096 INFO [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] regionserver.HStore(310): Store=11ddcd7bf8df7d1e6665f6c9cc6acd73/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:42,096 INFO [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:42,097 DEBUG [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2 2023-03-09 01:55:42,097 DEBUG [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2 2023-03-09 01:55:42,097 INFO [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11ddcd7bf8df7d1e6665f6c9cc6acd73 columnFamilyName family2 2023-03-09 01:55:42,099 INFO [StoreOpener-11ddcd7bf8df7d1e6665f6c9cc6acd73-1] regionserver.HStore(310): Store=11ddcd7bf8df7d1e6665f6c9cc6acd73/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:42,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:42,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:42,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:42,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:55:42,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:55:42,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 11ddcd7bf8df7d1e6665f6c9cc6acd73; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71276240, jitterRate=0.06209874153137207}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:42,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:42,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73., pid=110, masterSystemTime=1678326942088 2023-03-09 01:55:42,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:42,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:42,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:42,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fd53913779b3fcab65c638e819ce5953, NAME => 'ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953.', STARTKEY => '', ENDKEY => '1'} 2023-03-09 01:55:42,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:42,108 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=11ddcd7bf8df7d1e6665f6c9cc6acd73, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:42,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:42,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:42,109 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1678326942108"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326942108"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326942108"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326942108"}]},"ts":"1678326942108"} 2023-03-09 01:55:42,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:42,110 INFO [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:42,111 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-03-09 01:55:42,112 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure 11ddcd7bf8df7d1e6665f6c9cc6acd73, server=jenkins-hbase4.apache.org,46511,1678326819757 in 173 msec 2023-03-09 01:55:42,112 DEBUG [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1 2023-03-09 01:55:42,112 DEBUG [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1 2023-03-09 01:55:42,112 INFO [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd53913779b3fcab65c638e819ce5953 columnFamilyName family1 2023-03-09 01:55:42,113 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=11ddcd7bf8df7d1e6665f6c9cc6acd73, ASSIGN in 332 msec 2023-03-09 01:55:42,113 INFO [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] regionserver.HStore(310): Store=fd53913779b3fcab65c638e819ce5953/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:42,113 INFO [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:42,115 DEBUG [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2 2023-03-09 01:55:42,115 DEBUG [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2 2023-03-09 01:55:42,115 INFO [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd53913779b3fcab65c638e819ce5953 columnFamilyName family2 2023-03-09 01:55:42,116 INFO [StoreOpener-fd53913779b3fcab65c638e819ce5953-1] regionserver.HStore(310): Store=fd53913779b3fcab65c638e819ce5953/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:42,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:42,118 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/.tmp/family2/3be956f31fa14999bb6aece31a4f1f6c as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/3be956f31fa14999bb6aece31a4f1f6c 2023-03-09 01:55:42,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:42,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:42,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fd53913779b3fcab65c638e819ce5953 2023-03-09 01:55:42,124 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 00f768e6a69add6056570cdcb3ad746e/family2 of 00f768e6a69add6056570cdcb3ad746e into 3be956f31fa14999bb6aece31a4f1f6c(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:55:42,124 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:55:42,124 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e., storeName=00f768e6a69add6056570cdcb3ad746e/family2, priority=12, startTime=1678326939743; duration=0sec 2023-03-09 01:55:42,124 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:55:42,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:55:42,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fd53913779b3fcab65c638e819ce5953; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72358954, jitterRate=0.07823243737220764}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:42,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:42,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953., pid=109, masterSystemTime=1678326942088 2023-03-09 01:55:42,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:42,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:42,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:42,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b0d3ba46f6423b83104f4f0f94be0362, NAME => 'ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362.', STARTKEY => '2', ENDKEY => ''} 2023-03-09 01:55:42,127 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=fd53913779b3fcab65c638e819ce5953, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:42,127 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678326942126"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326942126"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326942126"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326942126"}]},"ts":"1678326942126"} 2023-03-09 01:55:42,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:42,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:55:42,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:42,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:42,128 INFO [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:42,129 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=106 2023-03-09 01:55:42,129 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=106, state=SUCCESS; OpenRegionProcedure fd53913779b3fcab65c638e819ce5953, server=jenkins-hbase4.apache.org,46511,1678326819757 in 191 msec 2023-03-09 01:55:42,130 DEBUG [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1 2023-03-09 01:55:42,130 DEBUG [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1 2023-03-09 01:55:42,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=fd53913779b3fcab65c638e819ce5953, ASSIGN in 349 msec 2023-03-09 01:55:42,130 INFO [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b0d3ba46f6423b83104f4f0f94be0362 columnFamilyName family1 2023-03-09 01:55:42,131 INFO [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] regionserver.HStore(310): Store=b0d3ba46f6423b83104f4f0f94be0362/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:42,131 INFO [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:42,133 DEBUG [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2 2023-03-09 01:55:42,133 DEBUG [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2 2023-03-09 01:55:42,133 INFO [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b0d3ba46f6423b83104f4f0f94be0362 columnFamilyName family2 2023-03-09 01:55:42,134 INFO [StoreOpener-b0d3ba46f6423b83104f4f0f94be0362-1] regionserver.HStore(310): Store=b0d3ba46f6423b83104f4f0f94be0362/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:55:42,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:42,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:42,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:55:42,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:55:42,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:55:42,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b0d3ba46f6423b83104f4f0f94be0362; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69137616, jitterRate=0.03023076057434082}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:55:42,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:42,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362., pid=111, masterSystemTime=1678326942088 2023-03-09 01:55:42,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:42,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:42,142 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=b0d3ba46f6423b83104f4f0f94be0362, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:42,142 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1678326942142"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326942142"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326942142"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326942142"}]},"ts":"1678326942142"} 2023-03-09 01:55:42,144 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-03-09 01:55:42,144 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure b0d3ba46f6423b83104f4f0f94be0362, server=jenkins-hbase4.apache.org,46511,1678326819757 in 206 msec 2023-03-09 01:55:42,146 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=108, resume processing ppid=105 2023-03-09 01:55:42,146 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=b0d3ba46f6423b83104f4f0f94be0362, ASSIGN in 364 msec 2023-03-09 01:55:42,146 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:55:42,147 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326942147"}]},"ts":"1678326942147"} 2023-03-09 01:55:42,148 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-09 01:55:42,151 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:55:42,152 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2060 sec 2023-03-09 01:55:42,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-51a937efd2f644babe8351499c9fa15a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-b54f40cf492f48f2a959cbed0a4fd66d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-8c204b7188634b12a4b18c8cb9e96815, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-6fd2b3985d2b4c7b812d559361310f38] to archive 2023-03-09 01:55:42,154 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:42,155 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-51a937efd2f644babe8351499c9fa15a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-51a937efd2f644babe8351499c9fa15a 2023-03-09 01:55:42,156 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-b54f40cf492f48f2a959cbed0a4fd66d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-b54f40cf492f48f2a959cbed0a4fd66d 2023-03-09 01:55:42,157 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-8c204b7188634b12a4b18c8cb9e96815 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-8c204b7188634b12a4b18c8cb9e96815 2023-03-09 01:55:42,158 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-6fd2b3985d2b4c7b812d559361310f38 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/family2/testCompactionWithHFileLink=00f768e6a69add6056570cdcb3ad746e-6fd2b3985d2b4c7b812d559361310f38 2023-03-09 01:55:43,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-09 01:55:43,054 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-09 01:55:43,060 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:43,061 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:43,061 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:43,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:43,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-09 01:55:43,065 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-09 01:55:43,065 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:43,065 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-09 01:55:43,065 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-09 01:55:43,065 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:43,065 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:43,066 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:43,066 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:43,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:43,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:43,067 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:43,067 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:43,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:43,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:43,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:43,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:43,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-09 01:55:43,068 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-09 01:55:43,068 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-09 01:55:43,068 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:43,068 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-09 01:55:43,068 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:43,068 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. started... 2023-03-09 01:55:43,070 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:43,070 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd53913779b3fcab65c638e819ce5953 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:55:43,070 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:43,070 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. started... 2023-03-09 01:55:43,070 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:43,070 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:43,070 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b0d3ba46f6423b83104f4f0f94be0362 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:55:43,070 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. started... 2023-03-09 01:55:43,071 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 11ddcd7bf8df7d1e6665f6c9cc6acd73 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:55:43,493 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309cc035d5b636c47159cc6ee24939b9599 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230309cc035d5b636c47159cc6ee24939b9599 2023-03-09 01:55:43,493 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309e8cf67a2cad84485a72b92897cfb07d2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230309e8cf67a2cad84485a72b92897cfb07d2 2023-03-09 01:55:43,493 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030981ec3e5899c64fb7a45d493a66a1d427 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023030981ec3e5899c64fb7a45d493a66a1d427 2023-03-09 01:55:43,899 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/17734c4655d042a1a67724101e89a5db 2023-03-09 01:55:43,899 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/9bf3e7969b9d431ea25a5a0fcb680e9b 2023-03-09 01:55:43,899 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/ce818a1cd34e4d938a36dcbe873f14ca 2023-03-09 01:55:43,917 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303095af912e2cbb9452e874f39d5ce669ed3 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303095af912e2cbb9452e874f39d5ce669ed3 2023-03-09 01:55:43,922 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/9ed3b1684ef74c6cb53b88183c0a785f 2023-03-09 01:55:43,925 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/17734c4655d042a1a67724101e89a5db as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/17734c4655d042a1a67724101e89a5db 2023-03-09 01:55:43,929 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/17734c4655d042a1a67724101e89a5db, entries=4, sequenceid=5, filesize=5.4 K 2023-03-09 01:55:43,929 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/9ed3b1684ef74c6cb53b88183c0a785f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/9ed3b1684ef74c6cb53b88183c0a785f 2023-03-09 01:55:43,932 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/9ed3b1684ef74c6cb53b88183c0a785f, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:55:43,933 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for fd53913779b3fcab65c638e819ce5953 in 863ms, sequenceid=5, compaction requested=false 2023-03-09 01:55:43,933 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-09 01:55:43,933 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:43,933 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:43,933 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:44,317 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309fa34300cec324745b09daa3ea800702c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230309fa34300cec324745b09daa3ea800702c 2023-03-09 01:55:44,317 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030922f12a868a7040688d5ec7f3565bc9c7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023030922f12a868a7040688d5ec7f3565bc9c7 2023-03-09 01:55:44,722 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/234a4ee5e6394d7388996b1dfe0915d7 2023-03-09 01:55:44,726 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/ce818a1cd34e4d938a36dcbe873f14ca as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/ce818a1cd34e4d938a36dcbe873f14ca 2023-03-09 01:55:44,730 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/f56c6746c87a4b8d94ce6ee3db1b1abd 2023-03-09 01:55:44,730 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/ce818a1cd34e4d938a36dcbe873f14ca, entries=4, sequenceid=5, filesize=5.4 K 2023-03-09 01:55:44,731 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/234a4ee5e6394d7388996b1dfe0915d7 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/234a4ee5e6394d7388996b1dfe0915d7 2023-03-09 01:55:44,734 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/9bf3e7969b9d431ea25a5a0fcb680e9b as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/9bf3e7969b9d431ea25a5a0fcb680e9b 2023-03-09 01:55:44,735 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/234a4ee5e6394d7388996b1dfe0915d7, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:55:44,736 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b0d3ba46f6423b83104f4f0f94be0362 in 1666ms, sequenceid=5, compaction requested=false 2023-03-09 01:55:44,737 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:44,737 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:44,737 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:44,739 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/9bf3e7969b9d431ea25a5a0fcb680e9b, entries=4, sequenceid=5, filesize=5.4 K 2023-03-09 01:55:44,739 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/f56c6746c87a4b8d94ce6ee3db1b1abd as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/f56c6746c87a4b8d94ce6ee3db1b1abd 2023-03-09 01:55:44,743 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/f56c6746c87a4b8d94ce6ee3db1b1abd, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:55:44,744 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 11ddcd7bf8df7d1e6665f6c9cc6acd73 in 1673ms, sequenceid=5, compaction requested=false 2023-03-09 01:55:44,744 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:44,744 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:44,744 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:44,744 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:44,744 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,744 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-09 01:55:44,744 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-09 01:55:44,746 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:44,746 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:44,748 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-09 01:55:44,748 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b4345da[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:44,748 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-09 01:55:44,748 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:44,750 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:44,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:44,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:44,750 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-09 01:55:44,750 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-09 01:55:44,750 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:44,750 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,750 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:44,752 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-09 01:55:44,752 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,752 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:44,752 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-09 01:55:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:44,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:44,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-09 01:55:44,754 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-09 01:55:44,754 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:44,754 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:44,754 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-09 01:55:44,754 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:44,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,756 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:44,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:44,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:44,757 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:44,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:44,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,758 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:44,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:44,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:44,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:44,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-09 01:55:44,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:44,761 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:44,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:44,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:44,761 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:44,762 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:44,762 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,862 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:44,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:44,864 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:44,864 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:44,864 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:44,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:44,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:44,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-09 01:55:44,869 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-09 01:55:44,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:44,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-09 01:55:44,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-09 01:55:44,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,869 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:44,871 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,871 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:44,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:44,871 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:44,872 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:44,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:44,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-09 01:55:44,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-09 01:55:44,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-09 01:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-09 01:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-09 01:55:44,873 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:44,873 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:44,873 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. started... 2023-03-09 01:55:44,873 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. started... 2023-03-09 01:55:44,873 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:44,873 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd53913779b3fcab65c638e819ce5953 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:55:44,873 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. started... 2023-03-09 01:55:44,873 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b0d3ba46f6423b83104f4f0f94be0362 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:55:44,873 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 11ddcd7bf8df7d1e6665f6c9cc6acd73 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:55:44,883 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309ab0beefa076742e3be26378ec319e3af to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230309ab0beefa076742e3be26378ec319e3af 2023-03-09 01:55:45,285 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309e8a4142664354273af00ba7be7b3dcf4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230309e8a4142664354273af00ba7be7b3dcf4 2023-03-09 01:55:45,285 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309cc4d65ae320241c9838ccef593b800e9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230309cc4d65ae320241c9838ccef593b800e9 2023-03-09 01:55:45,289 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/a4538c775bae4c59a50c82ce95088a42 2023-03-09 01:55:45,290 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/2dfa02f4ee8f4f67beac8d41bb5b4985 2023-03-09 01:55:45,690 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/c2e2151ba4674c0e96cbf5bb4b2ab490 2023-03-09 01:55:45,702 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030959cf44c5775a4655acb4c7a0198730cc to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030959cf44c5775a4655acb4c7a0198730cc 2023-03-09 01:55:45,703 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309444bb623f488489c934a0e01cf07e6fb to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230309444bb623f488489c934a0e01cf07e6fb 2023-03-09 01:55:45,704 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303098f9fdf9083bf471c944b4466f5aa3082 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303098f9fdf9083bf471c944b4466f5aa3082 2023-03-09 01:55:46,112 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/cfa3bb519ff64c15923d4b26ad3bd1f1 2023-03-09 01:55:46,112 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/af77c433b0c14e1ab7a6959285ad8624 2023-03-09 01:55:46,112 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/7d8163a3e3294d3899403bff96971015 2023-03-09 01:55:46,116 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/c2e2151ba4674c0e96cbf5bb4b2ab490 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/c2e2151ba4674c0e96cbf5bb4b2ab490 2023-03-09 01:55:46,117 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/a4538c775bae4c59a50c82ce95088a42 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/a4538c775bae4c59a50c82ce95088a42 2023-03-09 01:55:46,117 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/2dfa02f4ee8f4f67beac8d41bb5b4985 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/2dfa02f4ee8f4f67beac8d41bb5b4985 2023-03-09 01:55:46,122 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/a4538c775bae4c59a50c82ce95088a42, entries=4, sequenceid=9, filesize=5.4 K 2023-03-09 01:55:46,123 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/cfa3bb519ff64c15923d4b26ad3bd1f1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/cfa3bb519ff64c15923d4b26ad3bd1f1 2023-03-09 01:55:46,125 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/2dfa02f4ee8f4f67beac8d41bb5b4985, entries=4, sequenceid=9, filesize=5.4 K 2023-03-09 01:55:46,127 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/af77c433b0c14e1ab7a6959285ad8624 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/af77c433b0c14e1ab7a6959285ad8624 2023-03-09 01:55:46,127 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/c2e2151ba4674c0e96cbf5bb4b2ab490, entries=4, sequenceid=9, filesize=5.4 K 2023-03-09 01:55:46,128 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/cfa3bb519ff64c15923d4b26ad3bd1f1, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:55:46,128 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/7d8163a3e3294d3899403bff96971015 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/7d8163a3e3294d3899403bff96971015 2023-03-09 01:55:46,129 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 11ddcd7bf8df7d1e6665f6c9cc6acd73 in 1256ms, sequenceid=9, compaction requested=false 2023-03-09 01:55:46,129 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:46,129 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:46,129 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:46,131 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/af77c433b0c14e1ab7a6959285ad8624, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:55:46,132 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b0d3ba46f6423b83104f4f0f94be0362 in 1259ms, sequenceid=9, compaction requested=false 2023-03-09 01:55:46,132 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:46,132 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:46,132 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/7d8163a3e3294d3899403bff96971015, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:55:46,132 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:46,133 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for fd53913779b3fcab65c638e819ce5953 in 1260ms, sequenceid=9, compaction requested=false 2023-03-09 01:55:46,133 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:46,133 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:46,133 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:46,133 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:46,133 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,133 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-09 01:55:46,133 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-09 01:55:46,135 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:46,135 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:46,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:46,135 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:46,135 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:46,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:46,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:46,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:46,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:46,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-09 01:55:46,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63423010[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:46,137 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-09 01:55:46,137 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:46,138 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:46,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:46,139 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,139 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:46,139 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-09 01:55:46,139 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-09 01:55:46,139 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:46,144 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-09 01:55:46,144 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:46,144 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,144 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-09 01:55:46,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:46,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:46,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:46,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:46,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:46,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:46,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:46,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:46,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-09 01:55:46,147 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-09 01:55:46,147 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:46,147 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:46,147 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-09 01:55:46,147 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:46,148 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,148 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:46,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,148 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:46,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:46,149 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:46,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:46,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:46,150 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:46,153 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:46,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-09 01:55:46,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:46,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:46,153 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:46,154 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,154 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:46,254 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:46,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:46,256 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:46,256 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:46,256 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:46,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:46,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:46,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-09 01:55:46,260 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-09 01:55:46,260 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:46,260 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-09 01:55:46,260 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-09 01:55:46,261 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,261 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:46,263 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,263 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:46,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:46,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:46,263 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:46,263 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:46,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:46,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:46,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:46,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:46,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-09 01:55:46,264 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-09 01:55:46,264 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-09 01:55:46,264 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-09 01:55:46,264 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:46,264 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-09 01:55:46,265 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:46,265 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. started... 2023-03-09 01:55:46,265 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:46,265 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd53913779b3fcab65c638e819ce5953 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:55:46,265 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:46,265 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:46,265 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. started... 2023-03-09 01:55:46,265 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. started... 2023-03-09 01:55:46,265 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:46,265 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b0d3ba46f6423b83104f4f0f94be0362 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:55:46,265 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 11ddcd7bf8df7d1e6665f6c9cc6acd73 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:55:46,278 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309ee5688efbabb4d8389f5d3f79d5215ba to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230309ee5688efbabb4d8389f5d3f79d5215ba 2023-03-09 01:55:46,282 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/00f0e330adf949e4a7937a10ab5a8e42 2023-03-09 01:55:46,679 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303096ffae4358ee547af980b963d12de69f2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303096ffae4358ee547af980b963d12de69f2 2023-03-09 01:55:46,680 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303098f4c8c476d8344fd8fe0b8f608e925f2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303098f4c8c476d8344fd8fe0b8f608e925f2 2023-03-09 01:55:46,695 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303098cdf314bd2c845e5bbd2ce78fee3f15d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303098cdf314bd2c845e5bbd2ce78fee3f15d 2023-03-09 01:55:46,699 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/0520196cbfc0482c8e0084b19741d1f0 2023-03-09 01:55:46,702 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/00f0e330adf949e4a7937a10ab5a8e42 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/00f0e330adf949e4a7937a10ab5a8e42 2023-03-09 01:55:46,706 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/00f0e330adf949e4a7937a10ab5a8e42, entries=4, sequenceid=13, filesize=5.4 K 2023-03-09 01:55:46,706 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/0520196cbfc0482c8e0084b19741d1f0 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/0520196cbfc0482c8e0084b19741d1f0 2023-03-09 01:55:46,709 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/0520196cbfc0482c8e0084b19741d1f0, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:55:46,710 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 11ddcd7bf8df7d1e6665f6c9cc6acd73 in 445ms, sequenceid=13, compaction requested=true 2023-03-09 01:55:46,710 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:46,710 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:46,710 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:47,089 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/49e4fed6d402428e873709ac426ef09d 2023-03-09 01:55:47,089 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/f00cc5cb5ac44f0fb98f7183c5839ca8 2023-03-09 01:55:47,503 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309f6d38bc921ba45f4b68447be14a8fd4e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230309f6d38bc921ba45f4b68447be14a8fd4e 2023-03-09 01:55:47,503 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309074d75ee2e6f4c36ac879b12331969ca to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230309074d75ee2e6f4c36ac879b12331969ca 2023-03-09 01:55:47,593 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:55:47,910 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/8c309145464d40ccafe2d7dd4c2a7544 2023-03-09 01:55:47,910 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/37723562be8545d0916abb6ca8aa9f17 2023-03-09 01:55:47,914 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/f00cc5cb5ac44f0fb98f7183c5839ca8 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/f00cc5cb5ac44f0fb98f7183c5839ca8 2023-03-09 01:55:47,914 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/49e4fed6d402428e873709ac426ef09d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/49e4fed6d402428e873709ac426ef09d 2023-03-09 01:55:47,918 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/f00cc5cb5ac44f0fb98f7183c5839ca8, entries=4, sequenceid=13, filesize=5.4 K 2023-03-09 01:55:47,918 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/49e4fed6d402428e873709ac426ef09d, entries=4, sequenceid=13, filesize=5.4 K 2023-03-09 01:55:47,918 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/8c309145464d40ccafe2d7dd4c2a7544 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/8c309145464d40ccafe2d7dd4c2a7544 2023-03-09 01:55:47,918 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/37723562be8545d0916abb6ca8aa9f17 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/37723562be8545d0916abb6ca8aa9f17 2023-03-09 01:55:47,922 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/8c309145464d40ccafe2d7dd4c2a7544, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:55:47,922 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/37723562be8545d0916abb6ca8aa9f17, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:55:47,923 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for fd53913779b3fcab65c638e819ce5953 in 1658ms, sequenceid=13, compaction requested=true 2023-03-09 01:55:47,923 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b0d3ba46f6423b83104f4f0f94be0362 in 1658ms, sequenceid=13, compaction requested=true 2023-03-09 01:55:47,923 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:47,923 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:47,923 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:47,923 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:47,923 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:47,923 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:47,923 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:47,923 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,923 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-09 01:55:47,923 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-09 01:55:47,925 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,925 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:47,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:47,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:47,925 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:47,925 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:47,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:47,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:47,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:47,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:47,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-09 01:55:47,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@596d5c1[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:47,926 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-09 01:55:47,926 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:47,927 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:47,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:47,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:47,928 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-09 01:55:47,928 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-09 01:55:47,928 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:47,928 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,928 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:47,929 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,929 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-09 01:55:47,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,929 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:47,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:47,929 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-09 01:55:47,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:47,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:47,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:47,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:47,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:47,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:47,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:47,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-09 01:55:47,931 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-09 01:55:47,931 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:47,931 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:47,931 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-09 01:55:47,931 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:47,933 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:47,933 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:47,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:47,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:47,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:47,933 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:47,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:47,934 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:47,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:47,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:47,935 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:47,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:47,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:47,937 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:47,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:47,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:47,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-09 01:55:47,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:47,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:47,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:47,937 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:47,938 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:48,038 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:48,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:48,040 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:48,040 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:48,040 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:55:48,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:48,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:48,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-09 01:55:48,044 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-09 01:55:48,044 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:48,044 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-09 01:55:48,044 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-09 01:55:48,044 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:48,045 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:48,046 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:48,046 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:48,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:48,046 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:48,046 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:48,046 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:48,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:48,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:48,047 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-09 01:55:48,047 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-09 01:55:48,047 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-09 01:55:48,047 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-09 01:55:48,047 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:48,047 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-09 01:55:48,047 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:48,047 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. started... 2023-03-09 01:55:48,047 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:48,047 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:48,048 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd53913779b3fcab65c638e819ce5953 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:55:48,048 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:48,048 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. started... 2023-03-09 01:55:48,048 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:48,048 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. started... 2023-03-09 01:55:48,048 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 11ddcd7bf8df7d1e6665f6c9cc6acd73 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:55:48,048 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b0d3ba46f6423b83104f4f0f94be0362 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:55:48,062 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309bf1cbfb89fa949d9acd1a49ac866746e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230309bf1cbfb89fa949d9acd1a49ac866746e 2023-03-09 01:55:48,462 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309917b080c297948b9a2b34b78a29721e1 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230309917b080c297948b9a2b34b78a29721e1 2023-03-09 01:55:48,462 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030918b6584f253b4fae9654ea11f6d578b0 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023030918b6584f253b4fae9654ea11f6d578b0 2023-03-09 01:55:48,469 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/b259d690c8644f0b88eb8b4a0b9d236d 2023-03-09 01:55:48,867 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/556a6351ab9747819e177cc0753842c3 2023-03-09 01:55:48,867 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/a0b0677edb9848dc8cd1d4bd2fd7acba 2023-03-09 01:55:48,884 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309fa7ebdbcbab84c2892a0f29c824c9f14 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230309fa7ebdbcbab84c2892a0f29c824c9f14 2023-03-09 01:55:49,280 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303096d5876ade7724de58bcbd9bcd144e8d7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303096d5876ade7724de58bcbd9bcd144e8d7 2023-03-09 01:55:49,280 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030945378b4bb949465ca844030681e9d3eb to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023030945378b4bb949465ca844030681e9d3eb 2023-03-09 01:55:49,288 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/bb6c9500becf47948c0544b1d26d5b2b 2023-03-09 01:55:49,292 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/b259d690c8644f0b88eb8b4a0b9d236d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/b259d690c8644f0b88eb8b4a0b9d236d 2023-03-09 01:55:49,295 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/b259d690c8644f0b88eb8b4a0b9d236d, entries=4, sequenceid=17, filesize=5.4 K 2023-03-09 01:55:49,295 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/bb6c9500becf47948c0544b1d26d5b2b as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/bb6c9500becf47948c0544b1d26d5b2b 2023-03-09 01:55:49,298 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/bb6c9500becf47948c0544b1d26d5b2b, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:55:49,299 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b0d3ba46f6423b83104f4f0f94be0362 in 1251ms, sequenceid=17, compaction requested=true 2023-03-09 01:55:49,299 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:49,299 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:49,299 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:49,685 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/b655833114484771bc13862b3a37293c 2023-03-09 01:55:49,685 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/a2f2ff0f71964456a13c0788173f7283 2023-03-09 01:55:49,689 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/a0b0677edb9848dc8cd1d4bd2fd7acba as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/a0b0677edb9848dc8cd1d4bd2fd7acba 2023-03-09 01:55:49,689 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/556a6351ab9747819e177cc0753842c3 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/556a6351ab9747819e177cc0753842c3 2023-03-09 01:55:49,693 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/a0b0677edb9848dc8cd1d4bd2fd7acba, entries=4, sequenceid=17, filesize=5.4 K 2023-03-09 01:55:49,693 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/556a6351ab9747819e177cc0753842c3, entries=4, sequenceid=17, filesize=5.4 K 2023-03-09 01:55:49,693 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/b655833114484771bc13862b3a37293c as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/b655833114484771bc13862b3a37293c 2023-03-09 01:55:49,693 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/a2f2ff0f71964456a13c0788173f7283 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/a2f2ff0f71964456a13c0788173f7283 2023-03-09 01:55:49,698 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/b655833114484771bc13862b3a37293c, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:55:49,698 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/a2f2ff0f71964456a13c0788173f7283, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:55:49,698 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for fd53913779b3fcab65c638e819ce5953 in 1650ms, sequenceid=17, compaction requested=true 2023-03-09 01:55:49,698 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 11ddcd7bf8df7d1e6665f6c9cc6acd73 in 1650ms, sequenceid=17, compaction requested=true 2023-03-09 01:55:49,698 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:49,698 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:49,698 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:49,698 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:49,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:49,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:49,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:49,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-09 01:55:49,699 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-09 01:55:49,700 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,700 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:49,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:49,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:49,701 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:49,701 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:49,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:49,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:49,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:49,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:49,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-09 01:55:49,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25796909[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:49,702 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-09 01:55:49,702 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:49,703 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:49,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:49,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:49,703 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-09 01:55:49,703 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,703 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:49,703 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-09 01:55:49,703 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:49,705 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-09 01:55:49,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:49,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:49,705 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-09 01:55:49,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:49,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:49,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:49,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:49,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:49,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:49,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-09 01:55:49,707 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-09 01:55:49,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:49,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:49,707 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-09 01:55:49,707 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:49,710 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,710 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:49,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:49,710 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:49,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:49,711 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:49,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:49,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:49,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:49,712 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:49,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,714 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:49,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:49,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:49,714 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:49,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:49,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:49,715 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,715 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:49,715 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:49,715 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,715 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:49,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-09 01:55:49,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:49,715 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:49,715 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:49,715 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,715 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:49,716 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:49,816 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:49,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:49,827 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:49,838 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:49,848 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:49,849 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:49,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:55:49,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-09 01:55:49,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-09 01:55:49,876 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-09 01:55:49,876 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:49,876 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-09 01:55:49,876 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-09 01:55:49,877 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,877 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:49,882 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,882 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:49,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:49,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:49,882 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:49,882 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:55:49,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:49,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:49,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:55:49,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:49,883 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-09 01:55:49,883 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-09 01:55:49,883 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-09 01:55:49,883 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-09 01:55:49,883 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:55:49,883 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-09 01:55:49,883 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:55:49,883 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:49,883 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:49,883 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:55:49,883 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:49,883 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. started... 2023-03-09 01:55:49,883 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. started... 2023-03-09 01:55:49,884 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. started... 2023-03-09 01:55:49,884 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 11ddcd7bf8df7d1e6665f6c9cc6acd73 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:55:49,884 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b0d3ba46f6423b83104f4f0f94be0362 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:55:49,884 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fd53913779b3fcab65c638e819ce5953 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:55:50,305 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/b37259af9f31430e9c22cf2abb7aa404 2023-03-09 01:55:50,309 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37259af9f31430e9c22cf2abb7aa404 2023-03-09 01:55:50,704 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/bef71e958ed7408fbd08308900d085b8 2023-03-09 01:55:50,704 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/990fd2bd871947c8992b9fedb6ce6802 2023-03-09 01:55:50,707 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 990fd2bd871947c8992b9fedb6ce6802 2023-03-09 01:55:50,707 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bef71e958ed7408fbd08308900d085b8 2023-03-09 01:55:51,119 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/6c51b5b06c964f349256717db48cbb62 2023-03-09 01:55:51,122 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c51b5b06c964f349256717db48cbb62 2023-03-09 01:55:51,123 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/b37259af9f31430e9c22cf2abb7aa404 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/b37259af9f31430e9c22cf2abb7aa404 2023-03-09 01:55:51,126 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37259af9f31430e9c22cf2abb7aa404 2023-03-09 01:55:51,126 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/b37259af9f31430e9c22cf2abb7aa404, entries=3, sequenceid=23, filesize=5.0 K 2023-03-09 01:55:51,126 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/6c51b5b06c964f349256717db48cbb62 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/6c51b5b06c964f349256717db48cbb62 2023-03-09 01:55:51,129 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c51b5b06c964f349256717db48cbb62 2023-03-09 01:55:51,129 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/6c51b5b06c964f349256717db48cbb62, entries=1, sequenceid=23, filesize=5.0 K 2023-03-09 01:55:51,130 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for fd53913779b3fcab65c638e819ce5953 in 1246ms, sequenceid=23, compaction requested=true 2023-03-09 01:55:51,130 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:51,130 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:51,130 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:55:51,520 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/6b6fc63327c8403a8a0ea7ae1834642a 2023-03-09 01:55:51,520 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/5d1791727bb3481dab3ca771be0168be 2023-03-09 01:55:51,523 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b6fc63327c8403a8a0ea7ae1834642a 2023-03-09 01:55:51,523 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d1791727bb3481dab3ca771be0168be 2023-03-09 01:55:51,524 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/990fd2bd871947c8992b9fedb6ce6802 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/990fd2bd871947c8992b9fedb6ce6802 2023-03-09 01:55:51,524 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/bef71e958ed7408fbd08308900d085b8 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/bef71e958ed7408fbd08308900d085b8 2023-03-09 01:55:51,527 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 990fd2bd871947c8992b9fedb6ce6802 2023-03-09 01:55:51,527 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/990fd2bd871947c8992b9fedb6ce6802, entries=3, sequenceid=23, filesize=5.0 K 2023-03-09 01:55:51,527 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bef71e958ed7408fbd08308900d085b8 2023-03-09 01:55:51,527 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/bef71e958ed7408fbd08308900d085b8, entries=3, sequenceid=23, filesize=5.0 K 2023-03-09 01:55:51,528 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/6b6fc63327c8403a8a0ea7ae1834642a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/6b6fc63327c8403a8a0ea7ae1834642a 2023-03-09 01:55:51,528 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/5d1791727bb3481dab3ca771be0168be as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/5d1791727bb3481dab3ca771be0168be 2023-03-09 01:55:51,532 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b6fc63327c8403a8a0ea7ae1834642a 2023-03-09 01:55:51,532 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/6b6fc63327c8403a8a0ea7ae1834642a, entries=1, sequenceid=23, filesize=5.0 K 2023-03-09 01:55:51,532 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d1791727bb3481dab3ca771be0168be 2023-03-09 01:55:51,532 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/5d1791727bb3481dab3ca771be0168be, entries=1, sequenceid=23, filesize=5.0 K 2023-03-09 01:55:51,533 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b0d3ba46f6423b83104f4f0f94be0362 in 1649ms, sequenceid=23, compaction requested=true 2023-03-09 01:55:51,533 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 11ddcd7bf8df7d1e6665f6c9cc6acd73 in 1649ms, sequenceid=23, compaction requested=true 2023-03-09 01:55:51,533 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:51,533 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:51,533 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:51,533 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:51,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:55:51,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:55:51,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:55:51,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-09 01:55:51,533 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-09 01:55:51,535 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,535 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:51,535 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:51,535 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:55:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:51,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:51,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:51,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-09 01:55:51,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b585e9[Count = 0] remaining members to acquire global barrier 2023-03-09 01:55:51,537 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-09 01:55:51,537 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:51,539 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:51,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:51,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:51,539 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-09 01:55:51,539 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-09 01:55:51,539 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:55:51,539 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,539 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:55:51,541 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,541 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-09 01:55:51,541 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:51,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:51,541 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-09 01:55:51,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:51,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:51,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:51,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:51,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:51,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:51,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:55:51,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-09 01:55:51,543 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-09 01:55:51,543 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:55:51,543 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:55:51,543 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-09 01:55:51,543 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:55:51,550 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:51,550 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:51,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:51,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:55:51,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:51,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:51,550 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:51,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:55:51,551 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:51,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:51,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:55:51,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:51,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:51,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:55:51,552 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:51,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:55:51,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-09 01:55:51,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,554 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,554 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:55:51,554 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:55:51,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:55:51,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-09 01:55:51,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:55:51,554 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:55:51,554 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:51,555 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-09 01:55:51,555 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-09 01:55:51,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:55:51,555 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:55:51,555 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-09 01:55:51,555 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:51,555 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:55:51,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:55:51,555 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-09 01:55:51,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:55:51,555 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-09 01:55:51,655 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:55:51,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:55:51,656 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:51,656 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): fd53913779b3fcab65c638e819ce5953/family1 is initiating major compaction (all files) 2023-03-09 01:55:51,657 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:51,657 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:51,657 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of fd53913779b3fcab65c638e819ce5953/family1 in ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:51,657 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/17734c4655d042a1a67724101e89a5db, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/c2e2151ba4674c0e96cbf5bb4b2ab490, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/f00cc5cb5ac44f0fb98f7183c5839ca8, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/a0b0677edb9848dc8cd1d4bd2fd7acba, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/b37259af9f31430e9c22cf2abb7aa404] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp, totalSize=26.6 K 2023-03-09 01:55:51,657 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 17734c4655d042a1a67724101e89a5db, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326943060 2023-03-09 01:55:51,658 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting c2e2151ba4674c0e96cbf5bb4b2ab490, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326944864 2023-03-09 01:55:51,658 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting f00cc5cb5ac44f0fb98f7183c5839ca8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326946256 2023-03-09 01:55:51,659 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting a0b0677edb9848dc8cd1d4bd2fd7acba, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326948040 2023-03-09 01:55:51,659 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting b37259af9f31430e9c22cf2abb7aa404, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:55:51,669 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): fd53913779b3fcab65c638e819ce5953#family1#compaction#149 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:52,078 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309f888df4629c44e11a03cf9e96c6f3407_del 2023-03-09 01:55:52,078 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309f888df4629c44e11a03cf9e96c6f3407_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230309f888df4629c44e11a03cf9e96c6f3407_del 2023-03-09 01:55:52,891 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4075d9b5f3a14ec094e66f13bacd5757 2023-03-09 01:55:52,892 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family1/4075d9b5f3a14ec094e66f13bacd5757 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/4075d9b5f3a14ec094e66f13bacd5757 2023-03-09 01:55:52,896 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4075d9b5f3a14ec094e66f13bacd5757 2023-03-09 01:55:52,897 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fd53913779b3fcab65c638e819ce5953/family1 of fd53913779b3fcab65c638e819ce5953 into 4075d9b5f3a14ec094e66f13bacd5757(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:52,897 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:52,897 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:52,897 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): fd53913779b3fcab65c638e819ce5953/family2 is initiating major compaction (all files) 2023-03-09 01:55:52,897 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:52,897 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:52,898 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of fd53913779b3fcab65c638e819ce5953/family2 in ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:55:52,898 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/9ed3b1684ef74c6cb53b88183c0a785f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/7d8163a3e3294d3899403bff96971015, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/8c309145464d40ccafe2d7dd4c2a7544, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/b655833114484771bc13862b3a37293c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/6c51b5b06c964f349256717db48cbb62] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp, totalSize=25.3 K 2023-03-09 01:55:52,899 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 9ed3b1684ef74c6cb53b88183c0a785f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326943060 2023-03-09 01:55:52,899 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 7d8163a3e3294d3899403bff96971015, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326944864 2023-03-09 01:55:52,900 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 8c309145464d40ccafe2d7dd4c2a7544, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326946256 2023-03-09 01:55:52,900 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting b655833114484771bc13862b3a37293c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326948040 2023-03-09 01:55:52,900 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 6c51b5b06c964f349256717db48cbb62, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:55:52,909 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): fd53913779b3fcab65c638e819ce5953#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:52,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/17734c4655d042a1a67724101e89a5db, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/c2e2151ba4674c0e96cbf5bb4b2ab490, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/f00cc5cb5ac44f0fb98f7183c5839ca8, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/a0b0677edb9848dc8cd1d4bd2fd7acba, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/b37259af9f31430e9c22cf2abb7aa404] to archive 2023-03-09 01:55:52,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:52,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/17734c4655d042a1a67724101e89a5db to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/17734c4655d042a1a67724101e89a5db 2023-03-09 01:55:52,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/c2e2151ba4674c0e96cbf5bb4b2ab490 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/c2e2151ba4674c0e96cbf5bb4b2ab490 2023-03-09 01:55:52,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/f00cc5cb5ac44f0fb98f7183c5839ca8 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/f00cc5cb5ac44f0fb98f7183c5839ca8 2023-03-09 01:55:52,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/a0b0677edb9848dc8cd1d4bd2fd7acba to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/a0b0677edb9848dc8cd1d4bd2fd7acba 2023-03-09 01:55:52,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/b37259af9f31430e9c22cf2abb7aa404 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/b37259af9f31430e9c22cf2abb7aa404 2023-03-09 01:55:53,317 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309790de4c9dee1438d821bf3fd9f42585b_del 2023-03-09 01:55:53,317 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309790de4c9dee1438d821bf3fd9f42585b_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230309790de4c9dee1438d821bf3fd9f42585b_del 2023-03-09 01:55:54,130 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cff202b3265e442fad65c3b19024cd84 2023-03-09 01:55:54,131 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/.tmp/family2/cff202b3265e442fad65c3b19024cd84 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/cff202b3265e442fad65c3b19024cd84 2023-03-09 01:55:54,134 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cff202b3265e442fad65c3b19024cd84 2023-03-09 01:55:54,135 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fd53913779b3fcab65c638e819ce5953/family2 of fd53913779b3fcab65c638e819ce5953 into cff202b3265e442fad65c3b19024cd84(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:54,135 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:54,135 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:54,135 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): b0d3ba46f6423b83104f4f0f94be0362/family1 is initiating major compaction (all files) 2023-03-09 01:55:54,136 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:54,136 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:54,136 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of b0d3ba46f6423b83104f4f0f94be0362/family1 in ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:54,136 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/ce818a1cd34e4d938a36dcbe873f14ca, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/2dfa02f4ee8f4f67beac8d41bb5b4985, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/49e4fed6d402428e873709ac426ef09d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/b259d690c8644f0b88eb8b4a0b9d236d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/990fd2bd871947c8992b9fedb6ce6802] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp, totalSize=26.6 K 2023-03-09 01:55:54,136 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting ce818a1cd34e4d938a36dcbe873f14ca, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326943061 2023-03-09 01:55:54,137 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 2dfa02f4ee8f4f67beac8d41bb5b4985, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326944864 2023-03-09 01:55:54,137 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 49e4fed6d402428e873709ac426ef09d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326946256 2023-03-09 01:55:54,137 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting b259d690c8644f0b88eb8b4a0b9d236d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326948040 2023-03-09 01:55:54,137 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 990fd2bd871947c8992b9fedb6ce6802, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:55:54,145 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): b0d3ba46f6423b83104f4f0f94be0362#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:54,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/9ed3b1684ef74c6cb53b88183c0a785f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/7d8163a3e3294d3899403bff96971015, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/8c309145464d40ccafe2d7dd4c2a7544, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/b655833114484771bc13862b3a37293c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/6c51b5b06c964f349256717db48cbb62] to archive 2023-03-09 01:55:54,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:54,151 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/9ed3b1684ef74c6cb53b88183c0a785f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/9ed3b1684ef74c6cb53b88183c0a785f 2023-03-09 01:55:54,152 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/7d8163a3e3294d3899403bff96971015 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/7d8163a3e3294d3899403bff96971015 2023-03-09 01:55:54,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/8c309145464d40ccafe2d7dd4c2a7544 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/8c309145464d40ccafe2d7dd4c2a7544 2023-03-09 01:55:54,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/b655833114484771bc13862b3a37293c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/b655833114484771bc13862b3a37293c 2023-03-09 01:55:54,154 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/6c51b5b06c964f349256717db48cbb62 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family2/6c51b5b06c964f349256717db48cbb62 2023-03-09 01:55:54,552 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230309866f866d1a8d4bacb3c127f959d2e0be_del 2023-03-09 01:55:54,552 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309866f866d1a8d4bacb3c127f959d2e0be_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230309866f866d1a8d4bacb3c127f959d2e0be_del 2023-03-09 01:55:55,366 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 64f848e7b4a6486e99f2e0b23841d808 2023-03-09 01:55:55,367 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family1/64f848e7b4a6486e99f2e0b23841d808 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/64f848e7b4a6486e99f2e0b23841d808 2023-03-09 01:55:55,370 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 64f848e7b4a6486e99f2e0b23841d808 2023-03-09 01:55:55,371 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b0d3ba46f6423b83104f4f0f94be0362/family1 of b0d3ba46f6423b83104f4f0f94be0362 into 64f848e7b4a6486e99f2e0b23841d808(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:55,371 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:55,371 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:55,371 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): b0d3ba46f6423b83104f4f0f94be0362/family2 is initiating major compaction (all files) 2023-03-09 01:55:55,372 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:55,372 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:55,373 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of b0d3ba46f6423b83104f4f0f94be0362/family2 in ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:55:55,373 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/234a4ee5e6394d7388996b1dfe0915d7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/af77c433b0c14e1ab7a6959285ad8624, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/37723562be8545d0916abb6ca8aa9f17, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/bb6c9500becf47948c0544b1d26d5b2b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/6b6fc63327c8403a8a0ea7ae1834642a] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp, totalSize=25.3 K 2023-03-09 01:55:55,373 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 234a4ee5e6394d7388996b1dfe0915d7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326943061 2023-03-09 01:55:55,373 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting af77c433b0c14e1ab7a6959285ad8624, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326944864 2023-03-09 01:55:55,374 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 37723562be8545d0916abb6ca8aa9f17, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326946256 2023-03-09 01:55:55,374 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting bb6c9500becf47948c0544b1d26d5b2b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326948040 2023-03-09 01:55:55,374 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 6b6fc63327c8403a8a0ea7ae1834642a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:55:55,383 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): b0d3ba46f6423b83104f4f0f94be0362#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:55,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/ce818a1cd34e4d938a36dcbe873f14ca, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/2dfa02f4ee8f4f67beac8d41bb5b4985, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/49e4fed6d402428e873709ac426ef09d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/b259d690c8644f0b88eb8b4a0b9d236d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/990fd2bd871947c8992b9fedb6ce6802] to archive 2023-03-09 01:55:55,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:55,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/ce818a1cd34e4d938a36dcbe873f14ca to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/ce818a1cd34e4d938a36dcbe873f14ca 2023-03-09 01:55:55,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/2dfa02f4ee8f4f67beac8d41bb5b4985 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/2dfa02f4ee8f4f67beac8d41bb5b4985 2023-03-09 01:55:55,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/49e4fed6d402428e873709ac426ef09d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/49e4fed6d402428e873709ac426ef09d 2023-03-09 01:55:55,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/b259d690c8644f0b88eb8b4a0b9d236d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/b259d690c8644f0b88eb8b4a0b9d236d 2023-03-09 01:55:55,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/990fd2bd871947c8992b9fedb6ce6802 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/990fd2bd871947c8992b9fedb6ce6802 2023-03-09 01:55:55,794 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023030905e8063780624bb1a87205cf533f933c_del 2023-03-09 01:55:55,794 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030905e8063780624bb1a87205cf533f933c_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023030905e8063780624bb1a87205cf533f933c_del 2023-03-09 01:55:56,608 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d205565d55a14822ba887a70f628a7b7 2023-03-09 01:55:56,609 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/.tmp/family2/d205565d55a14822ba887a70f628a7b7 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/d205565d55a14822ba887a70f628a7b7 2023-03-09 01:55:56,612 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d205565d55a14822ba887a70f628a7b7 2023-03-09 01:55:56,613 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b0d3ba46f6423b83104f4f0f94be0362/family2 of b0d3ba46f6423b83104f4f0f94be0362 into d205565d55a14822ba887a70f628a7b7(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:56,613 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:55:56,613 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:56,614 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 is initiating major compaction (all files) 2023-03-09 01:55:56,614 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:56,614 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:56,615 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 in ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:56,615 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/9bf3e7969b9d431ea25a5a0fcb680e9b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/a4538c775bae4c59a50c82ce95088a42, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/00f0e330adf949e4a7937a10ab5a8e42, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/556a6351ab9747819e177cc0753842c3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/bef71e958ed7408fbd08308900d085b8] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp, totalSize=26.6 K 2023-03-09 01:55:56,615 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 9bf3e7969b9d431ea25a5a0fcb680e9b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326943061 2023-03-09 01:55:56,616 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting a4538c775bae4c59a50c82ce95088a42, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326944864 2023-03-09 01:55:56,616 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 00f0e330adf949e4a7937a10ab5a8e42, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326946256 2023-03-09 01:55:56,617 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 556a6351ab9747819e177cc0753842c3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326948040 2023-03-09 01:55:56,617 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting bef71e958ed7408fbd08308900d085b8, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:55:56,625 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 11ddcd7bf8df7d1e6665f6c9cc6acd73#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:56,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/234a4ee5e6394d7388996b1dfe0915d7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/af77c433b0c14e1ab7a6959285ad8624, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/37723562be8545d0916abb6ca8aa9f17, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/bb6c9500becf47948c0544b1d26d5b2b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/6b6fc63327c8403a8a0ea7ae1834642a] to archive 2023-03-09 01:55:56,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:56,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/234a4ee5e6394d7388996b1dfe0915d7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/234a4ee5e6394d7388996b1dfe0915d7 2023-03-09 01:55:56,654 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/af77c433b0c14e1ab7a6959285ad8624 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/af77c433b0c14e1ab7a6959285ad8624 2023-03-09 01:55:56,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/37723562be8545d0916abb6ca8aa9f17 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/37723562be8545d0916abb6ca8aa9f17 2023-03-09 01:55:56,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/bb6c9500becf47948c0544b1d26d5b2b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/bb6c9500becf47948c0544b1d26d5b2b 2023-03-09 01:55:56,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/6b6fc63327c8403a8a0ea7ae1834642a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family2/6b6fc63327c8403a8a0ea7ae1834642a 2023-03-09 01:55:57,035 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230309983487668737492aaa40817f2e976ee1_del 2023-03-09 01:55:57,036 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309983487668737492aaa40817f2e976ee1_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230309983487668737492aaa40817f2e976ee1_del 2023-03-09 01:55:57,849 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 28b0d54af68d401b85fd45034d47d01f 2023-03-09 01:55:57,850 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family1/28b0d54af68d401b85fd45034d47d01f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/28b0d54af68d401b85fd45034d47d01f 2023-03-09 01:55:57,853 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 28b0d54af68d401b85fd45034d47d01f 2023-03-09 01:55:57,854 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 of 11ddcd7bf8df7d1e6665f6c9cc6acd73 into 28b0d54af68d401b85fd45034d47d01f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:57,854 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:57,854 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:55:57,854 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 11ddcd7bf8df7d1e6665f6c9cc6acd73/family2 is initiating major compaction (all files) 2023-03-09 01:55:57,854 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:55:57,855 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:55:57,857 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 11ddcd7bf8df7d1e6665f6c9cc6acd73/family2 in ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:55:57,857 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/f56c6746c87a4b8d94ce6ee3db1b1abd, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/cfa3bb519ff64c15923d4b26ad3bd1f1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/0520196cbfc0482c8e0084b19741d1f0, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/a2f2ff0f71964456a13c0788173f7283, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/5d1791727bb3481dab3ca771be0168be] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp, totalSize=25.3 K 2023-03-09 01:55:57,857 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting f56c6746c87a4b8d94ce6ee3db1b1abd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326943061 2023-03-09 01:55:57,857 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting cfa3bb519ff64c15923d4b26ad3bd1f1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326944864 2023-03-09 01:55:57,858 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 0520196cbfc0482c8e0084b19741d1f0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326946256 2023-03-09 01:55:57,858 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting a2f2ff0f71964456a13c0788173f7283, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326948040 2023-03-09 01:55:57,858 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 5d1791727bb3481dab3ca771be0168be, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:55:57,868 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 11ddcd7bf8df7d1e6665f6c9cc6acd73#family2#compaction#154 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:55:57,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/9bf3e7969b9d431ea25a5a0fcb680e9b, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/a4538c775bae4c59a50c82ce95088a42, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/00f0e330adf949e4a7937a10ab5a8e42, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/556a6351ab9747819e177cc0753842c3, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/bef71e958ed7408fbd08308900d085b8] to archive 2023-03-09 01:55:57,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:57,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/9bf3e7969b9d431ea25a5a0fcb680e9b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/9bf3e7969b9d431ea25a5a0fcb680e9b 2023-03-09 01:55:57,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/a4538c775bae4c59a50c82ce95088a42 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/a4538c775bae4c59a50c82ce95088a42 2023-03-09 01:55:57,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/00f0e330adf949e4a7937a10ab5a8e42 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/00f0e330adf949e4a7937a10ab5a8e42 2023-03-09 01:55:57,958 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/556a6351ab9747819e177cc0753842c3 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/556a6351ab9747819e177cc0753842c3 2023-03-09 01:55:57,959 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/bef71e958ed7408fbd08308900d085b8 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/bef71e958ed7408fbd08308900d085b8 2023-03-09 01:55:58,281 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023030997c8c654467b473ea9062945cce13c6c_del 2023-03-09 01:55:58,282 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030997c8c654467b473ea9062945cce13c6c_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023030997c8c654467b473ea9062945cce13c6c_del 2023-03-09 01:55:59,097 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b4e543664d3e4687bf83575c64b91fa6 2023-03-09 01:55:59,098 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/.tmp/family2/b4e543664d3e4687bf83575c64b91fa6 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/b4e543664d3e4687bf83575c64b91fa6 2023-03-09 01:55:59,102 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b4e543664d3e4687bf83575c64b91fa6 2023-03-09 01:55:59,102 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 11ddcd7bf8df7d1e6665f6c9cc6acd73/family2 of 11ddcd7bf8df7d1e6665f6c9cc6acd73 into b4e543664d3e4687bf83575c64b91fa6(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:55:59,103 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:55:59,106 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:59,108 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:59,111 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:55:59,112 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:55:59,116 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-09 01:55:59,129 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-09 01:55:59,132 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309f888df4629c44e11a03cf9e96c6f3407_del 2023-03-09 01:55:59,135 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230309983487668737492aaa40817f2e976ee1_del 2023-03-09 01:55:59,138 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230309866f866d1a8d4bacb3c127f959d2e0be_del 2023-03-09 01:55:59,138 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-09 01:55:59,139 DEBUG [Listener at localhost/33973] zookeeper.ReadOnlyZKClient(139): Connect 0x4f6c2fc6 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:55:59,143 DEBUG [Listener at localhost/33973] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5282e94f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:55:59,145 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230309 for table='ns:testMinorCompaction' and column='family1' 2023-03-09 01:55:59,145 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230309 for table='ns:testMinorCompaction' and column='family1' 2023-03-09 01:55:59,145 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230309 for table='ns:testMinorCompaction' and column='family1' 2023-03-09 01:55:59,151 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/f56c6746c87a4b8d94ce6ee3db1b1abd, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/cfa3bb519ff64c15923d4b26ad3bd1f1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/0520196cbfc0482c8e0084b19741d1f0, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/a2f2ff0f71964456a13c0788173f7283, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/5d1791727bb3481dab3ca771be0168be] to archive 2023-03-09 01:55:59,151 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:59,156 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/f56c6746c87a4b8d94ce6ee3db1b1abd to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/f56c6746c87a4b8d94ce6ee3db1b1abd 2023-03-09 01:55:59,160 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/cfa3bb519ff64c15923d4b26ad3bd1f1 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/cfa3bb519ff64c15923d4b26ad3bd1f1 2023-03-09 01:55:59,164 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/0520196cbfc0482c8e0084b19741d1f0 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/0520196cbfc0482c8e0084b19741d1f0 2023-03-09 01:55:59,165 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/a2f2ff0f71964456a13c0788173f7283 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/a2f2ff0f71964456a13c0788173f7283 2023-03-09 01:55:59,167 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/5d1791727bb3481dab3ca771be0168be to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family2/5d1791727bb3481dab3ca771be0168be 2023-03-09 01:55:59,586 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030928e73368c2db4ddaaf3e1580f54020ee to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023030928e73368c2db4ddaaf3e1580f54020ee 2023-03-09 01:55:59,587 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='ns:testMinorCompaction' column='family1' 2023-03-09 01:55:59,599 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:55:59,602 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37362, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:55:59,609 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:55:59,612 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53844, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:55:59,628 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230309/family1/1d7fdbb4e727487b8393fd9c56e48bc8 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-09 01:55:59,629 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230309/family1/1d7fdbb4e727487b8393fd9c56e48bc8}] 2023-03-09 01:55:59,635 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230309/family1/1d7fdbb4e727487b8393fd9c56e48bc8 for inclusion in fd53913779b3fcab65c638e819ce5953/family1 2023-03-09 01:55:59,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-09 01:55:59,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last=1 2023-03-09 01:55:59,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:55:59,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230309/family1/1d7fdbb4e727487b8393fd9c56e48bc8 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__uh8ahpntsuukmsv8329c24297kertqc20iqp0hcsbmnel4vhhce8nnifub38aenf/family1/1d7fdbb4e727487b8393fd9c56e48bc8 2023-03-09 01:55:59,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__uh8ahpntsuukmsv8329c24297kertqc20iqp0hcsbmnel4vhhce8nnifub38aenf/family1/1d7fdbb4e727487b8393fd9c56e48bc8 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/efb7bfc89db543b1824af066380ed09f_SeqId_28_ 2023-03-09 01:55:59,642 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__uh8ahpntsuukmsv8329c24297kertqc20iqp0hcsbmnel4vhhce8nnifub38aenf/family1/1d7fdbb4e727487b8393fd9c56e48bc8 into fd53913779b3fcab65c638e819ce5953/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/efb7bfc89db543b1824af066380ed09f_SeqId_28_ - updating store file list. 2023-03-09 01:55:59,646 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/efb7bfc89db543b1824af066380ed09f_SeqId_28_ into fd53913779b3fcab65c638e819ce5953/family1 2023-03-09 01:55:59,647 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__uh8ahpntsuukmsv8329c24297kertqc20iqp0hcsbmnel4vhhce8nnifub38aenf/family1/1d7fdbb4e727487b8393fd9c56e48bc8 into fd53913779b3fcab65c638e819ce5953/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/family1/efb7bfc89db543b1824af066380ed09f_SeqId_28_) 2023-03-09 01:55:59,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__uh8ahpntsuukmsv8329c24297kertqc20iqp0hcsbmnel4vhhce8nnifub38aenf/family1/1d7fdbb4e727487b8393fd9c56e48bc8 2023-03-09 01:55:59,649 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='ns:testMinorCompaction' column='family1' 2023-03-09 01:55:59,650 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:55:59,651 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230309e8cf67a2cad84485a72b92897cfb07d2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230309e8cf67a2cad84485a72b92897cfb07d2 2023-03-09 01:55:59,651 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-09 01:55:59,651 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230309' 2023-03-09 01:55:59,651 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230309 for table='ns:testMinorCompaction' and column='family1' 2023-03-09 01:55:59,985 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303091ec8d6faaffa456a84ef9a0f57fe1b57 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303091ec8d6faaffa456a84ef9a0f57fe1b57 2023-03-09 01:55:59,986 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230309 table='ns:testMinorCompaction' column='family1' 2023-03-09 01:55:59,989 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309ffc684ce42db4110bcc6377622576d7c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230309ffc684ce42db4110bcc6377622576d7c 2023-03-09 01:55:59,990 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='ns:testMinorCompaction' column='family1' 2023-03-09 01:56:00,010 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230309/family1/c90f509b517c4144acf48e8c92ff01dc first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-09 01:56:00,010 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230309/family1/c90f509b517c4144acf48e8c92ff01dc}] 2023-03-09 01:56:00,013 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230309/family1/c90f509b517c4144acf48e8c92ff01dc for inclusion in 11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 2023-03-09 01:56:00,015 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230309/family1/86130e18999443418f1f70f0e1439a6f first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-09 01:56:00,016 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230309/family1/86130e18999443418f1f70f0e1439a6f}] 2023-03-09 01:56:00,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-09 01:56:00,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-09 01:56:00,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:56:00,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230309/family1/c90f509b517c4144acf48e8c92ff01dc to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__armk1ngikiq7eaa27gqam8qs7h6phujglpgqjofrsrr66q8npokrvo1vh9ja770v/family1/c90f509b517c4144acf48e8c92ff01dc 2023-03-09 01:56:00,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230309/family1/86130e18999443418f1f70f0e1439a6f for inclusion in b0d3ba46f6423b83104f4f0f94be0362/family1 2023-03-09 01:56:00,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__armk1ngikiq7eaa27gqam8qs7h6phujglpgqjofrsrr66q8npokrvo1vh9ja770v/family1/c90f509b517c4144acf48e8c92ff01dc as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/d2cace9fa6954b769a44564e33f1d652_SeqId_28_ 2023-03-09 01:56:00,021 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__armk1ngikiq7eaa27gqam8qs7h6phujglpgqjofrsrr66q8npokrvo1vh9ja770v/family1/c90f509b517c4144acf48e8c92ff01dc into 11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/d2cace9fa6954b769a44564e33f1d652_SeqId_28_ - updating store file list. 2023-03-09 01:56:00,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-09 01:56:00,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-09 01:56:00,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:56:00,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230309/family1/86130e18999443418f1f70f0e1439a6f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__8kas93urf81ns0ipdq4puhq0s0udi8oibu6mis2qnan8qokdmom2i8k5g3b0k61k/family1/86130e18999443418f1f70f0e1439a6f 2023-03-09 01:56:00,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/d2cace9fa6954b769a44564e33f1d652_SeqId_28_ into 11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 2023-03-09 01:56:00,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__armk1ngikiq7eaa27gqam8qs7h6phujglpgqjofrsrr66q8npokrvo1vh9ja770v/family1/c90f509b517c4144acf48e8c92ff01dc into 11ddcd7bf8df7d1e6665f6c9cc6acd73/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/family1/d2cace9fa6954b769a44564e33f1d652_SeqId_28_) 2023-03-09 01:56:00,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__armk1ngikiq7eaa27gqam8qs7h6phujglpgqjofrsrr66q8npokrvo1vh9ja770v/family1/c90f509b517c4144acf48e8c92ff01dc 2023-03-09 01:56:00,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__8kas93urf81ns0ipdq4puhq0s0udi8oibu6mis2qnan8qokdmom2i8k5g3b0k61k/family1/86130e18999443418f1f70f0e1439a6f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/bcd1503e7c4048e38c6579a94068dded_SeqId_28_ 2023-03-09 01:56:00,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__8kas93urf81ns0ipdq4puhq0s0udi8oibu6mis2qnan8qokdmom2i8k5g3b0k61k/family1/86130e18999443418f1f70f0e1439a6f into b0d3ba46f6423b83104f4f0f94be0362/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/bcd1503e7c4048e38c6579a94068dded_SeqId_28_ - updating store file list. 2023-03-09 01:56:00,026 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230309 table='ns:testMinorCompaction' column='family1' 2023-03-09 01:56:00,026 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:00,028 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230309cc035d5b636c47159cc6ee24939b9599 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230309cc035d5b636c47159cc6ee24939b9599 2023-03-09 01:56:00,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/bcd1503e7c4048e38c6579a94068dded_SeqId_28_ into b0d3ba46f6423b83104f4f0f94be0362/family1 2023-03-09 01:56:00,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__8kas93urf81ns0ipdq4puhq0s0udi8oibu6mis2qnan8qokdmom2i8k5g3b0k61k/family1/86130e18999443418f1f70f0e1439a6f into b0d3ba46f6423b83104f4f0f94be0362/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/family1/bcd1503e7c4048e38c6579a94068dded_SeqId_28_) 2023-03-09 01:56:00,028 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-09 01:56:00,028 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230309' 2023-03-09 01:56:00,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__ns_testMinorCompaction__8kas93urf81ns0ipdq4puhq0s0udi8oibu6mis2qnan8qokdmom2i8k5g3b0k61k/family1/86130e18999443418f1f70f0e1439a6f 2023-03-09 01:56:00,028 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230309 for table='ns:testMinorCompaction' and column='family1' 2023-03-09 01:56:00,030 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='ns:testMinorCompaction' column='family1' 2023-03-09 01:56:00,030 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:00,032 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023030981ec3e5899c64fb7a45d493a66a1d427 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023030981ec3e5899c64fb7a45d493a66a1d427 2023-03-09 01:56:00,032 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-09 01:56:00,032 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230309' 2023-03-09 01:56:00,032 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230309 for table='ns:testMinorCompaction' and column='family1' 2023-03-09 01:56:00,032 INFO [Listener at localhost/33973] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-09 01:56:00,035 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:56:00,037 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:56:00,039 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:56:00,040 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:56:00,060 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=254 (was 281), OpenFileDescriptor=636 (was 745), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=137 (was 142), ProcessCount=170 (was 171), AvailableMemoryMB=7935 (was 7950) 2023-03-09 01:56:00,073 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=254, OpenFileDescriptor=636, MaxFileDescriptor=60000, SystemLoadAverage=137, ProcessCount=170, AvailableMemoryMB=7934 2023-03-09 01:56:00,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-09 01:56:00,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:00,077 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:56:00,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-09 01:56:00,078 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:56:00,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-09 01:56:00,079 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:00,080 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce empty. 2023-03-09 01:56:00,080 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:00,080 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-09 01:56:00,166 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:56:00,166 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5289509b to 127.0.0.1:50548 2023-03-09 01:56:00,166 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:00,166 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:56:00,166 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4f6c2fc6 to 127.0.0.1:50548 2023-03-09 01:56:00,167 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:00,167 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:56:00,167 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x683e65d4 to 127.0.0.1:50548 2023-03-09 01:56:00,167 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:00,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-09 01:56:00,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-09 01:56:00,487 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-09 01:56:00,488 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => f0e8e486b536a2c5f3a4c2d719d586ce, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:56:00,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-09 01:56:00,894 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:56:00,894 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing f0e8e486b536a2c5f3a4c2d719d586ce, disabling compactions & flushes 2023-03-09 01:56:00,894 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:00,894 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:00,894 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. after waiting 0 ms 2023-03-09 01:56:00,894 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:00,894 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:00,894 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:00,896 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:56:00,897 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678326960896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326960896"}]},"ts":"1678326960896"} 2023-03-09 01:56:00,898 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-09 01:56:00,898 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:56:00,899 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326960899"}]},"ts":"1678326960899"} 2023-03-09 01:56:00,900 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-09 01:56:00,903 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f0e8e486b536a2c5f3a4c2d719d586ce, ASSIGN}] 2023-03-09 01:56:00,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f0e8e486b536a2c5f3a4c2d719d586ce, ASSIGN 2023-03-09 01:56:00,904 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f0e8e486b536a2c5f3a4c2d719d586ce, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:56:01,055 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=f0e8e486b536a2c5f3a4c2d719d586ce, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:01,056 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678326961055"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326961055"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326961055"}]},"ts":"1678326961055"} 2023-03-09 01:56:01,057 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure f0e8e486b536a2c5f3a4c2d719d586ce, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:56:01,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-09 01:56:01,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:01,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f0e8e486b536a2c5f3a4c2d719d586ce, NAME => 'testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce.', STARTKEY => '', ENDKEY => ''} 2023-03-09 01:56:01,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:01,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:56:01,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:01,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:01,213 INFO [StoreOpener-f0e8e486b536a2c5f3a4c2d719d586ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:01,214 DEBUG [StoreOpener-f0e8e486b536a2c5f3a4c2d719d586ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1 2023-03-09 01:56:01,214 DEBUG [StoreOpener-f0e8e486b536a2c5f3a4c2d719d586ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1 2023-03-09 01:56:01,215 INFO [StoreOpener-f0e8e486b536a2c5f3a4c2d719d586ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0e8e486b536a2c5f3a4c2d719d586ce columnFamilyName family1 2023-03-09 01:56:01,216 INFO [StoreOpener-f0e8e486b536a2c5f3a4c2d719d586ce-1] regionserver.HStore(310): Store=f0e8e486b536a2c5f3a4c2d719d586ce/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:56:01,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:01,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:01,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:01,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:56:01,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f0e8e486b536a2c5f3a4c2d719d586ce; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69805298, jitterRate=0.04017999768257141}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-09 01:56:01,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:01,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce., pid=114, masterSystemTime=1678326961209 2023-03-09 01:56:01,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:01,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:01,223 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=f0e8e486b536a2c5f3a4c2d719d586ce, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:01,223 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1678326961223"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326961223"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326961223"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326961223"}]},"ts":"1678326961223"} 2023-03-09 01:56:01,225 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-09 01:56:01,226 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure f0e8e486b536a2c5f3a4c2d719d586ce, server=jenkins-hbase4.apache.org,46511,1678326819757 in 167 msec 2023-03-09 01:56:01,227 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-09 01:56:01,227 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=f0e8e486b536a2c5f3a4c2d719d586ce, ASSIGN in 322 msec 2023-03-09 01:56:01,227 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:56:01,227 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326961227"}]},"ts":"1678326961227"} 2023-03-09 01:56:01,228 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-09 01:56:01,230 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:56:01,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1550 sec 2023-03-09 01:56:02,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-09 01:56:02,183 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-09 01:56:02,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:02,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:02,203 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:02,203 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:02,203 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:02,203 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:02,204 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:02,204 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:02,206 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:02,206 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:02,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:02,207 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:02,207 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:02,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:02,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:02,208 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:02,208 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:02,208 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:02,208 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:02,208 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:02,208 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:02,208 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:02,208 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:02,209 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-09 01:56:02,633 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130881dc0903549477b8abdfeb5f7c51297 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130881dc0903549477b8abdfeb5f7c51297 2023-03-09 01:56:03,037 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/0750ad694cbf49a691ea1dcbfc1a331d 2023-03-09 01:56:03,041 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/0750ad694cbf49a691ea1dcbfc1a331d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0750ad694cbf49a691ea1dcbfc1a331d 2023-03-09 01:56:03,044 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0750ad694cbf49a691ea1dcbfc1a331d, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-09 01:56:03,045 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 836ms, sequenceid=5, compaction requested=false 2023-03-09 01:56:03,045 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:03,046 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:03,046 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:03,046 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:03,046 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:03,046 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,046 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:03,046 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:03,048 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,048 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:03,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:03,048 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,048 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:03,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:03,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:03,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7685f7df[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:03,050 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:03,050 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,051 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:03,051 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:03,051 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:03,052 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:03,052 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,052 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:03,053 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:03,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:03,055 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:03,055 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:03,055 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:03,055 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:03,055 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,055 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:03,057 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,057 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,057 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:03,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,057 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:03,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:03,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:03,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,058 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:03,061 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:03,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:03,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:03,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:03,061 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,161 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:03,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:03,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:03,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:03,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,167 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:03,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:03,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:03,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:03,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,168 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,169 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,169 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:03,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:03,170 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,170 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:03,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,171 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:03,171 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:03,171 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:03,171 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:03,171 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:03,171 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:03,171 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:03,171 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-09 01:56:03,582 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c6bdcfcd0f1f43ea8a4db5d4d9facc83 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128c6bdcfcd0f1f43ea8a4db5d4d9facc83 2023-03-09 01:56:03,987 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/7f64dffdd7fc47ebb78bd30869af5b04 2023-03-09 01:56:03,990 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/7f64dffdd7fc47ebb78bd30869af5b04 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/7f64dffdd7fc47ebb78bd30869af5b04 2023-03-09 01:56:03,993 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/7f64dffdd7fc47ebb78bd30869af5b04, entries=1, sequenceid=9, filesize=5.0 K 2023-03-09 01:56:03,994 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 823ms, sequenceid=9, compaction requested=false 2023-03-09 01:56:03,995 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:03,995 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:03,995 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:03,995 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:03,995 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,995 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:03,995 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:03,997 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,997 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:03,997 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:03,997 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:03,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:03,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@458e0c13[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:03,999 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:03,999 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,000 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,000 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:04,000 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:04,000 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:04,000 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,000 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:04,001 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:04,001 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:04,001 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,001 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:04,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:04,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:04,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:04,003 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:04,003 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:04,003 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:04,003 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,003 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:04,006 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,006 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:04,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,006 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:04,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:04,006 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,008 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:04,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,010 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:04,010 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:04,010 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:04,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:04,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:04,010 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:04,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:04,010 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:56:04,010 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:04,010 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:04,010 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:04,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:04,011 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,011 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,011 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,011 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,111 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:04,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:04,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:04,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:04,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,116 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:04,116 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:04,116 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:04,116 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:04,117 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,117 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,126 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,126 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:04,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:04,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:04,126 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,126 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:04,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,127 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:04,128 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:04,128 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:04,128 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:56:04,539 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201017520f1dbcc4593963f81864e1f0501 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201017520f1dbcc4593963f81864e1f0501 2023-03-09 01:56:04,945 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/5e2d7c53093c49e2b91d56a7ea75d0ab 2023-03-09 01:56:04,948 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/5e2d7c53093c49e2b91d56a7ea75d0ab as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/5e2d7c53093c49e2b91d56a7ea75d0ab 2023-03-09 01:56:04,951 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/5e2d7c53093c49e2b91d56a7ea75d0ab, entries=1, sequenceid=13, filesize=4.9 K 2023-03-09 01:56:04,952 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 824ms, sequenceid=13, compaction requested=true 2023-03-09 01:56:04,952 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:04,952 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:04,952 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:04,952 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:04,952 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,953 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:04,953 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:04,955 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,955 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:04,955 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,955 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:04,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:04,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:04,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bd1048a[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:04,956 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:04,956 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,958 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:04,958 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,958 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:04,958 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:04,960 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,960 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:04,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:04,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:04,960 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:04,960 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:04,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:04,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:04,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:04,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:04,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:04,962 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:04,962 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:04,962 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:04,962 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,962 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:04,964 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,964 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:04,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:04,964 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:04,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:04,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:04,964 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:04,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:04,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,965 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:04,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:04,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:04,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:04,969 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:04,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:04,969 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:04,970 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,070 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:05,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:05,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:05,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:05,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,075 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:05,076 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:05,076 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:05,076 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:05,076 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,076 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,078 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,078 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:05,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:05,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:05,078 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,078 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:05,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:05,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,079 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,079 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:05,079 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,079 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:05,079 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:05,079 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:05,079 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:05,079 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:05,079 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:05,079 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:05,079 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:56:05,492 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120527fcdbd872784ad19999d9050ee26e16 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120527fcdbd872784ad19999d9050ee26e16 2023-03-09 01:56:05,897 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/3b60955e95054e18b4f6a953c9e1b9ad 2023-03-09 01:56:05,901 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/3b60955e95054e18b4f6a953c9e1b9ad as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/3b60955e95054e18b4f6a953c9e1b9ad 2023-03-09 01:56:05,904 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/3b60955e95054e18b4f6a953c9e1b9ad, entries=1, sequenceid=17, filesize=4.9 K 2023-03-09 01:56:05,905 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 826ms, sequenceid=17, compaction requested=true 2023-03-09 01:56:05,905 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:05,905 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:05,905 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:05,905 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:05,905 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,905 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:05,905 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:05,908 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,908 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:05,908 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,908 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:05,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:05,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e346d4[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:05,910 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:05,910 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,915 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,915 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,915 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:05,916 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:05,915 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:05,916 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:05,921 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:05,921 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,921 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:05,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:05,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:05,921 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:05,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:05,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:05,923 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:05,923 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:05,923 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:05,923 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,923 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:05,925 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,925 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:05,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:05,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,925 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:05,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:05,925 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:05,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:05,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:05,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,926 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:05,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,929 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,929 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:05,929 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:05,929 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:05,929 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,929 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:05,930 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:05,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:05,930 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:05,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:05,930 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,930 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,930 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:05,930 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:05,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:05,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:05,930 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:06,030 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:06,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:06,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:06,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:06,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,036 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:06,036 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:06,036 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:06,036 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:06,037 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,037 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,038 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,038 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:06,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:06,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:06,038 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,038 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:06,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:06,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,039 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,039 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:06,039 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,039 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:06,039 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:06,039 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:06,039 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:06,040 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:06,040 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:06,040 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:06,040 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:56:06,451 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512286baf7c4d1a744b6d8bb9226b0ab8f199 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512286baf7c4d1a744b6d8bb9226b0ab8f199 2023-03-09 01:56:06,747 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:56:06,856 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/68dd64a2807d451bba14fffc377f4362 2023-03-09 01:56:06,861 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/68dd64a2807d451bba14fffc377f4362 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/68dd64a2807d451bba14fffc377f4362 2023-03-09 01:56:06,864 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/68dd64a2807d451bba14fffc377f4362, entries=1, sequenceid=21, filesize=4.9 K 2023-03-09 01:56:06,865 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 825ms, sequenceid=21, compaction requested=true 2023-03-09 01:56:06,865 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:06,865 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:06,865 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:06,865 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:06,865 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,865 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:06,865 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:06,868 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,868 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:06,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:06,868 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:06,868 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:06,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:06,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:06,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:06,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a017278[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:06,870 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:06,870 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,871 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,871 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:06,871 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:06,871 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:06,872 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,872 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:06,873 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:06,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:06,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:06,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:06,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:06,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:06,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:06,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:06,876 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:06,876 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:06,876 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:06,876 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,876 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:06,878 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,878 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,878 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:06,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:06,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:06,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:06,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:06,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,880 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,881 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:06,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,887 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:06,887 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:06,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:06,887 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:06,887 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:06,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:06,887 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,887 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:06,887 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,887 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:06,888 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:06,888 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:06,888 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,888 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:06,988 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:06,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:06,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:06,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:06,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,994 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:06,994 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:06,994 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:06,994 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:06,995 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,995 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,996 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:06,996 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:06,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:06,997 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:06,997 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:06,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:06,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,998 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,998 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:06,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:06,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:06,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:06,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:06,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:06,998 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:06,998 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:06,998 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:06,998 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:56:07,410 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f46613d102da421ea774ddaed83faf42 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231f46613d102da421ea774ddaed83faf42 2023-03-09 01:56:07,816 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/ce3cdd82f6a64810a8c18d08e7872467 2023-03-09 01:56:07,819 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/ce3cdd82f6a64810a8c18d08e7872467 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/ce3cdd82f6a64810a8c18d08e7872467 2023-03-09 01:56:07,822 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/ce3cdd82f6a64810a8c18d08e7872467, entries=1, sequenceid=25, filesize=4.9 K 2023-03-09 01:56:07,823 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 825ms, sequenceid=25, compaction requested=true 2023-03-09 01:56:07,823 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:07,823 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:07,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:07,825 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,825 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:07,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:07,826 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,826 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:07,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:07,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:07,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:07,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:07,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f5a13bf[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:07,827 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:07,827 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,829 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,829 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:07,829 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:07,829 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,829 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:07,829 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:07,830 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:07,830 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,830 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:07,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:07,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:07,831 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:07,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:07,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:07,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:07,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:07,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:07,833 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:07,833 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:07,833 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:07,833 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,833 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:07,834 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,834 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:07,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:07,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,834 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:07,835 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:07,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,836 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,838 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:07,838 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:07,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:07,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:07,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:07,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:07,838 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:07,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:07,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:07,939 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:07,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:07,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:07,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:07,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,945 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:07,945 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:07,945 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:07,945 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:07,945 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,945 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,947 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,947 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:07,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:07,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:07,947 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:07,947 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:07,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:07,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,948 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:07,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:07,948 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:07,948 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:07,948 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:07,949 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:07,949 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:07,949 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:07,949 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:07,949 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:56:07,958 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010158a47264bd3c494ababebc7256c11665 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010158a47264bd3c494ababebc7256c11665 2023-03-09 01:56:08,363 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/31171ab4779a481eafb37229d5d2b01c 2023-03-09 01:56:08,367 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/31171ab4779a481eafb37229d5d2b01c as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/31171ab4779a481eafb37229d5d2b01c 2023-03-09 01:56:08,370 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/31171ab4779a481eafb37229d5d2b01c, entries=1, sequenceid=29, filesize=4.9 K 2023-03-09 01:56:08,371 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 422ms, sequenceid=29, compaction requested=true 2023-03-09 01:56:08,371 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:08,371 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:08,371 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:08,371 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:08,371 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,371 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:08,371 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:08,373 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,373 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:08,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:08,373 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,373 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:08,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:08,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:08,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:08,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:08,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c8cbf8a[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:08,374 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:08,374 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,375 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,376 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:08,376 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:08,376 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:08,376 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,376 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:08,377 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:08,377 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:08,377 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,377 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:08,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:08,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:08,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:08,379 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:08,379 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:08,379 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:08,379 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,379 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:08,381 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,381 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:08,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:08,381 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:08,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,382 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:08,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:08,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:08,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:08,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,383 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:08,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:08,385 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:08,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:08,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:08,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:08,385 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:56:08,385 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:08,386 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:08,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:08,486 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:08,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:08,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:08,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:08,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,492 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:08,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:08,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:08,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:08,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,492 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,494 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,494 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:08,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:08,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:08,494 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:08,494 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:08,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,495 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,495 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:08,495 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:08,495 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:08,495 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:08,495 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:08,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:08,496 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:08,496 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:08,496 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:08,496 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:56:08,907 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103443d12b8f9314df3bc87fbedc446c77b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103443d12b8f9314df3bc87fbedc446c77b 2023-03-09 01:56:09,312 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/d1655f9eb6c4491e856d57410f331628 2023-03-09 01:56:09,315 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/d1655f9eb6c4491e856d57410f331628 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/d1655f9eb6c4491e856d57410f331628 2023-03-09 01:56:09,318 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/d1655f9eb6c4491e856d57410f331628, entries=1, sequenceid=33, filesize=4.9 K 2023-03-09 01:56:09,319 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 823ms, sequenceid=33, compaction requested=true 2023-03-09 01:56:09,319 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:09,319 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:09,319 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:09,319 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:09,319 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,319 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:09,319 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:09,321 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,321 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:09,321 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,321 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:09,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:09,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55408529[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:09,322 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:09,322 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,323 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,323 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:09,323 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:09,323 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,323 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:09,323 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:09,326 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,326 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:09,326 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:09,326 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:09,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:09,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:09,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:09,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:09,328 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:09,328 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:09,328 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:09,328 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,328 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:09,332 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,332 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:09,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:09,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,332 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:09,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:09,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:09,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:09,332 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:09,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,333 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:09,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,335 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:09,335 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:09,335 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:09,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:09,336 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:09,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:09,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:09,335 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,336 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,336 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,336 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,336 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,336 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:09,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:09,336 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:09,336 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:09,436 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:09,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:09,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:09,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:09,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,442 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:09,442 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:09,442 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:09,442 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:09,443 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,443 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,444 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:09,444 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:09,445 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:09,445 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:09,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:09,446 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:09,446 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:09,446 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:09,446 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:09,446 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:09,446 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:09,446 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:09,446 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:56:09,857 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079c1fa1bdb1a944f7afa9317d7cb07851 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509079c1fa1bdb1a944f7afa9317d7cb07851 2023-03-09 01:56:10,262 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/95190b3ec6934bbfbf2b7418d6819ded 2023-03-09 01:56:10,265 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/95190b3ec6934bbfbf2b7418d6819ded as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/95190b3ec6934bbfbf2b7418d6819ded 2023-03-09 01:56:10,268 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/95190b3ec6934bbfbf2b7418d6819ded, entries=1, sequenceid=37, filesize=4.9 K 2023-03-09 01:56:10,269 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 823ms, sequenceid=37, compaction requested=true 2023-03-09 01:56:10,269 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:10,269 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:10,269 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:10,269 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:10,269 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,269 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:10,269 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,271 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,271 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:10,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:10,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:10,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:10,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:10,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@403af4b[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:10,273 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:10,273 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,274 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:10,274 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:10,274 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:10,274 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,274 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:10,275 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:10,275 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:10,275 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,275 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:10,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:10,278 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:10,278 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:10,278 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:10,278 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,278 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:10,280 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,280 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:10,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:10,280 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:10,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,280 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:10,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:10,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:10,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:10,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,282 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:10,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,283 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,283 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:10,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:10,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:10,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:10,284 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:10,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:10,284 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,384 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:10,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:10,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:10,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:10,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,390 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:10,391 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:10,391 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-09 01:56:10,391 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-09 01:56:10,391 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,391 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,392 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:10,392 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:10,393 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,393 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,393 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,393 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,393 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-09 01:56:10,393 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,394 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-09 01:56:10,394 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:10,394 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-09 01:56:10,394 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-09 01:56:10,394 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:10,394 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:10,394 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. started... 2023-03-09 01:56:10,394 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f0e8e486b536a2c5f3a4c2d719d586ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-09 01:56:10,804 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120df64a61f1d564825b7aa90dff97f27d7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120df64a61f1d564825b7aa90dff97f27d7 2023-03-09 01:56:10,808 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/1979c5061a9b40f88d41683557e77bb1 2023-03-09 01:56:10,812 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/1979c5061a9b40f88d41683557e77bb1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/1979c5061a9b40f88d41683557e77bb1 2023-03-09 01:56:10,815 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/1979c5061a9b40f88d41683557e77bb1, entries=1, sequenceid=41, filesize=4.9 K 2023-03-09 01:56:10,816 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f0e8e486b536a2c5f3a4c2d719d586ce in 422ms, sequenceid=41, compaction requested=true 2023-03-09 01:56:10,816 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:10,816 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:10,816 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-09 01:56:10,816 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-09 01:56:10,816 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,816 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-09 01:56:10,816 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-09 01:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,818 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,818 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:10,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:10,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:10,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:10,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-09 01:56:10,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b7b939f[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:10,820 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-09 01:56:10,820 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,822 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,822 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-09 01:56:10,822 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-09 01:56:10,822 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:10,822 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,822 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:10,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-09 01:56:10,823 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:10,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,823 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-09 01:56:10,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:10,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:10,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:10,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:10,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:10,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:10,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-09 01:56:10,826 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-09 01:56:10,826 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:10,826 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:10,826 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,826 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:10,828 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,828 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,828 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:10,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:10,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:10,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:10,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:10,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:10,828 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:10,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,829 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:10,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,832 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:10,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-09 01:56:10,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-09 01:56:10,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:10,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:10,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:10,832 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,833 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,933 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:10,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:10,935 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-09 01:56:10,936 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:56:10,937 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:56:10,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:56:10,938 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327570938 2023-03-09 01:56:10,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-09 01:56:10,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-09 01:56:10,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:56:10,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x39ea9093 to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:56:10,945 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@184b8e6f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:56:10,946 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:56:11,770 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c21e973a09b74c8b863c1d6fa73c65b8 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103c21e973a09b74c8b863c1d6fa73c65b8 2023-03-09 01:56:11,771 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:56:11,783 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:56:11,786 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49710, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:56:11,791 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:56:11,792 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57402, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:56:11,800 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d987d4b991624b7abc6070059e69adbc first=Optional[r5] last=Optional[r6] 2023-03-09 01:56:11,801 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d987d4b991624b7abc6070059e69adbc}] 2023-03-09 01:56:11,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d987d4b991624b7abc6070059e69adbc for inclusion in f0e8e486b536a2c5f3a4c2d719d586ce/family1 2023-03-09 01:56:11,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-09 01:56:11,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:56:11,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:11,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/d987d4b991624b7abc6070059e69adbc to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__mj2mstu0vadtgp1k9vmah22hmcs8indtec8md2rvdagjlipdjicp42tahiou69bo/family1/d987d4b991624b7abc6070059e69adbc 2023-03-09 01:56:11,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__mj2mstu0vadtgp1k9vmah22hmcs8indtec8md2rvdagjlipdjicp42tahiou69bo/family1/d987d4b991624b7abc6070059e69adbc as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/407463fd99464df88b260336d253985c_SeqId_44_ 2023-03-09 01:56:11,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__mj2mstu0vadtgp1k9vmah22hmcs8indtec8md2rvdagjlipdjicp42tahiou69bo/family1/d987d4b991624b7abc6070059e69adbc into f0e8e486b536a2c5f3a4c2d719d586ce/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/407463fd99464df88b260336d253985c_SeqId_44_ - updating store file list. 2023-03-09 01:56:11,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/407463fd99464df88b260336d253985c_SeqId_44_ into f0e8e486b536a2c5f3a4c2d719d586ce/family1 2023-03-09 01:56:11,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__mj2mstu0vadtgp1k9vmah22hmcs8indtec8md2rvdagjlipdjicp42tahiou69bo/family1/d987d4b991624b7abc6070059e69adbc into f0e8e486b536a2c5f3a4c2d719d586ce/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/407463fd99464df88b260336d253985c_SeqId_44_) 2023-03-09 01:56:11,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__mj2mstu0vadtgp1k9vmah22hmcs8indtec8md2rvdagjlipdjicp42tahiou69bo/family1/d987d4b991624b7abc6070059e69adbc 2023-03-09 01:56:11,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-09 01:56:11,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-09 01:56:11,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): f0e8e486b536a2c5f3a4c2d719d586ce/family1 is initiating minor compaction 2023-03-09 01:56:11,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2da5296a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:56:11,815 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f0e8e486b536a2c5f3a4c2d719d586ce/family1 in testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:11,815 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/7f64dffdd7fc47ebb78bd30869af5b04, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/5e2d7c53093c49e2b91d56a7ea75d0ab, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/3b60955e95054e18b4f6a953c9e1b9ad, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/68dd64a2807d451bba14fffc377f4362, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/ce3cdd82f6a64810a8c18d08e7872467, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/31171ab4779a481eafb37229d5d2b01c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/d1655f9eb6c4491e856d57410f331628, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/95190b3ec6934bbfbf2b7418d6819ded, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/1979c5061a9b40f88d41683557e77bb1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/407463fd99464df88b260336d253985c_SeqId_44_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp, totalSize=49.7 K 2023-03-09 01:56:11,815 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 7f64dffdd7fc47ebb78bd30869af5b04, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-09 01:56:11,815 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 5e2d7c53093c49e2b91d56a7ea75d0ab, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-09 01:56:11,815 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 3b60955e95054e18b4f6a953c9e1b9ad, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-09 01:56:11,815 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 68dd64a2807d451bba14fffc377f4362, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-09 01:56:11,815 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting ce3cdd82f6a64810a8c18d08e7872467, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-09 01:56:11,815 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 31171ab4779a481eafb37229d5d2b01c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-09 01:56:11,815 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting d1655f9eb6c4491e856d57410f331628, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-09 01:56:11,816 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 95190b3ec6934bbfbf2b7418d6819ded, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-09 01:56:11,816 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 1979c5061a9b40f88d41683557e77bb1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-09 01:56:11,816 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 407463fd99464df88b260336d253985c_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-09 01:56:11,821 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:56:11,821 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:11,823 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010158a47264bd3c494ababebc7256c11665 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010158a47264bd3c494ababebc7256c11665 2023-03-09 01:56:11,824 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103443d12b8f9314df3bc87fbedc446c77b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103443d12b8f9314df3bc87fbedc446c77b 2023-03-09 01:56:11,824 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-09 01:56:11,824 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-09 01:56:11,824 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:56:11,824 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:56:11,840 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): f0e8e486b536a2c5f3a4c2d719d586ce#family1#compaction#165 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-03-09 01:56:12,263 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/0d2862ea0dc04e328f33e4f11c83f2de as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0d2862ea0dc04e328f33e4f11c83f2de 2023-03-09 01:56:12,267 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f0e8e486b536a2c5f3a4c2d719d586ce/family1 of f0e8e486b536a2c5f3a4c2d719d586ce into 0d2862ea0dc04e328f33e4f11c83f2de(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:56:12,267 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:12,268 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1678326971815; duration=0sec 2023-03-09 01:56:12,268 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:56:12,350 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/7f64dffdd7fc47ebb78bd30869af5b04, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/5e2d7c53093c49e2b91d56a7ea75d0ab, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/3b60955e95054e18b4f6a953c9e1b9ad, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/68dd64a2807d451bba14fffc377f4362, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/ce3cdd82f6a64810a8c18d08e7872467, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/31171ab4779a481eafb37229d5d2b01c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/d1655f9eb6c4491e856d57410f331628, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/95190b3ec6934bbfbf2b7418d6819ded, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/1979c5061a9b40f88d41683557e77bb1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/407463fd99464df88b260336d253985c_SeqId_44_] to archive 2023-03-09 01:56:12,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:12,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/7f64dffdd7fc47ebb78bd30869af5b04 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/7f64dffdd7fc47ebb78bd30869af5b04 2023-03-09 01:56:12,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/5e2d7c53093c49e2b91d56a7ea75d0ab to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/5e2d7c53093c49e2b91d56a7ea75d0ab 2023-03-09 01:56:12,355 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/3b60955e95054e18b4f6a953c9e1b9ad to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/3b60955e95054e18b4f6a953c9e1b9ad 2023-03-09 01:56:12,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/68dd64a2807d451bba14fffc377f4362 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/68dd64a2807d451bba14fffc377f4362 2023-03-09 01:56:12,357 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/ce3cdd82f6a64810a8c18d08e7872467 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/ce3cdd82f6a64810a8c18d08e7872467 2023-03-09 01:56:12,358 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/31171ab4779a481eafb37229d5d2b01c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/31171ab4779a481eafb37229d5d2b01c 2023-03-09 01:56:12,359 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/d1655f9eb6c4491e856d57410f331628 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/d1655f9eb6c4491e856d57410f331628 2023-03-09 01:56:12,360 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/95190b3ec6934bbfbf2b7418d6819ded to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/95190b3ec6934bbfbf2b7418d6819ded 2023-03-09 01:56:12,361 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/1979c5061a9b40f88d41683557e77bb1 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/1979c5061a9b40f88d41683557e77bb1 2023-03-09 01:56:12,361 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/407463fd99464df88b260336d253985c_SeqId_44_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/407463fd99464df88b260336d253985c_SeqId_44_ 2023-03-09 01:56:12,673 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d7b5c45221684dc18f8b6881821fd37b to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231d7b5c45221684dc18f8b6881821fd37b 2023-03-09 01:56:12,674 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:56:12,703 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/a475d9aba2a14a4e86f6695bf9a4469a first=Optional[r1] last=Optional[r4] 2023-03-09 01:56:12,704 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/a475d9aba2a14a4e86f6695bf9a4469a}] 2023-03-09 01:56:12,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/a475d9aba2a14a4e86f6695bf9a4469a for inclusion in f0e8e486b536a2c5f3a4c2d719d586ce/family1 2023-03-09 01:56:12,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-09 01:56:12,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:56:12,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:12,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/a475d9aba2a14a4e86f6695bf9a4469a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__s3o2fbrgborno6adpbaf6vajfq6c4289pl3i21hg9h9smib3v7jld55r9nusc3k4/family1/a475d9aba2a14a4e86f6695bf9a4469a 2023-03-09 01:56:12,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__s3o2fbrgborno6adpbaf6vajfq6c4289pl3i21hg9h9smib3v7jld55r9nusc3k4/family1/a475d9aba2a14a4e86f6695bf9a4469a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_ 2023-03-09 01:56:12,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__s3o2fbrgborno6adpbaf6vajfq6c4289pl3i21hg9h9smib3v7jld55r9nusc3k4/family1/a475d9aba2a14a4e86f6695bf9a4469a into f0e8e486b536a2c5f3a4c2d719d586ce/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_ - updating store file list. 2023-03-09 01:56:12,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_ into f0e8e486b536a2c5f3a4c2d719d586ce/family1 2023-03-09 01:56:12,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__s3o2fbrgborno6adpbaf6vajfq6c4289pl3i21hg9h9smib3v7jld55r9nusc3k4/family1/a475d9aba2a14a4e86f6695bf9a4469a into f0e8e486b536a2c5f3a4c2d719d586ce/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_) 2023-03-09 01:56:12,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__s3o2fbrgborno6adpbaf6vajfq6c4289pl3i21hg9h9smib3v7jld55r9nusc3k4/family1/a475d9aba2a14a4e86f6695bf9a4469a 2023-03-09 01:56:12,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-09 01:56:12,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-09 01:56:12,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(1912): f0e8e486b536a2c5f3a4c2d719d586ce/family1 is initiating minor compaction (all files) 2023-03-09 01:56:12,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@576738ce; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:56:12,716 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f0e8e486b536a2c5f3a4c2d719d586ce/family1 in testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:12,716 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0750ad694cbf49a691ea1dcbfc1a331d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0d2862ea0dc04e328f33e4f11c83f2de, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp, totalSize=178.1 K 2023-03-09 01:56:12,717 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 0750ad694cbf49a691ea1dcbfc1a331d, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878843359 2023-03-09 01:56:12,717 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting 0d2862ea0dc04e328f33e4f11c83f2de, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621243359 2023-03-09 01:56:12,717 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] compactions.Compactor(207): Compacting bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965243359 2023-03-09 01:56:12,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:56:12,718 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:12,720 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201017520f1dbcc4593963f81864e1f0501 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201017520f1dbcc4593963f81864e1f0501 2023-03-09 01:56:12,721 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120527fcdbd872784ad19999d9050ee26e16 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120527fcdbd872784ad19999d9050ee26e16 2023-03-09 01:56:12,722 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512286baf7c4d1a744b6d8bb9226b0ab8f199 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512286baf7c4d1a744b6d8bb9226b0ab8f199 2023-03-09 01:56:12,723 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231f46613d102da421ea774ddaed83faf42 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231f46613d102da421ea774ddaed83faf42 2023-03-09 01:56:12,723 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-09 01:56:12,723 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-09 01:56:12,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:56:12,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:56:12,733 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] throttle.PressureAwareThroughputController(145): f0e8e486b536a2c5f3a4c2d719d586ce#family1#compaction#166 average throughput is 17.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-03-09 01:56:13,566 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/.tmp/family1/ec3f3a7289114b72b66f2d4dfc30a187 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/ec3f3a7289114b72b66f2d4dfc30a187 2023-03-09 01:56:13,570 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a638d2393b2f4dc7a8313dc0b45ca5cb to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130a638d2393b2f4dc7a8313dc0b45ca5cb 2023-03-09 01:56:13,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:56:13,573 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f0e8e486b536a2c5f3a4c2d719d586ce/family1 of f0e8e486b536a2c5f3a4c2d719d586ce into ec3f3a7289114b72b66f2d4dfc30a187(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:56:13,573 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:13,573 INFO [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1678326972716; duration=0sec 2023-03-09 01:56:13,573 DEBUG [RS:0;jenkins-hbase4:46511-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-09 01:56:13,603 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/97bf2428c05f4188bf30417898f45e68 first=Optional[r00] last=Optional[r8] 2023-03-09 01:56:13,604 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/97bf2428c05f4188bf30417898f45e68}] 2023-03-09 01:56:13,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/97bf2428c05f4188bf30417898f45e68 for inclusion in f0e8e486b536a2c5f3a4c2d719d586ce/family1 2023-03-09 01:56:13,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-09 01:56:13,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last= 2023-03-09 01:56:13,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:13,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/97bf2428c05f4188bf30417898f45e68 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c3eugvddoij631u0r4c7j0nnimvf0sk40ep5a88djsr40aba3nvnkof2hlkuamhl/family1/97bf2428c05f4188bf30417898f45e68 2023-03-09 01:56:13,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c3eugvddoij631u0r4c7j0nnimvf0sk40ep5a88djsr40aba3nvnkof2hlkuamhl/family1/97bf2428c05f4188bf30417898f45e68 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/b113bb4b79404813bb70bfd733c7ea46_SeqId_50_ 2023-03-09 01:56:13,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c3eugvddoij631u0r4c7j0nnimvf0sk40ep5a88djsr40aba3nvnkof2hlkuamhl/family1/97bf2428c05f4188bf30417898f45e68 into f0e8e486b536a2c5f3a4c2d719d586ce/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/b113bb4b79404813bb70bfd733c7ea46_SeqId_50_ - updating store file list. 2023-03-09 01:56:13,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/b113bb4b79404813bb70bfd733c7ea46_SeqId_50_ into f0e8e486b536a2c5f3a4c2d719d586ce/family1 2023-03-09 01:56:13,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c3eugvddoij631u0r4c7j0nnimvf0sk40ep5a88djsr40aba3nvnkof2hlkuamhl/family1/97bf2428c05f4188bf30417898f45e68 into f0e8e486b536a2c5f3a4c2d719d586ce/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/b113bb4b79404813bb70bfd733c7ea46_SeqId_50_) 2023-03-09 01:56:13,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c3eugvddoij631u0r4c7j0nnimvf0sk40ep5a88djsr40aba3nvnkof2hlkuamhl/family1/97bf2428c05f4188bf30417898f45e68 2023-03-09 01:56:13,619 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:56:13,620 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:13,621 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120df64a61f1d564825b7aa90dff97f27d7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120df64a61f1d564825b7aa90dff97f27d7 2023-03-09 01:56:13,622 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128c6bdcfcd0f1f43ea8a4db5d4d9facc83 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128c6bdcfcd0f1f43ea8a4db5d4d9facc83 2023-03-09 01:56:13,623 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130881dc0903549477b8abdfeb5f7c51297 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130881dc0903549477b8abdfeb5f7c51297 2023-03-09 01:56:13,623 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-09 01:56:13,623 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-09 01:56:13,626 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:56:13,626 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:56:13,626 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-09 01:56:13,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-09 01:56:13,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:56:13,629 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-09 01:56:13,630 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.6940 sec 2023-03-09 01:56:13,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0750ad694cbf49a691ea1dcbfc1a331d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0d2862ea0dc04e328f33e4f11c83f2de, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_] to archive 2023-03-09 01:56:13,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:13,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0750ad694cbf49a691ea1dcbfc1a331d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0750ad694cbf49a691ea1dcbfc1a331d 2023-03-09 01:56:13,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0d2862ea0dc04e328f33e4f11c83f2de to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/0d2862ea0dc04e328f33e4f11c83f2de 2023-03-09 01:56:13,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_ to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/family1/bc5acce673f84ba68ac2df853c42b9ea_SeqId_47_ 2023-03-09 01:56:13,846 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=250 (was 254), OpenFileDescriptor=635 (was 636), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=144 (was 137) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=7921 (was 7934) 2023-03-09 01:56:13,858 INFO [Listener at localhost/33973] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=250, OpenFileDescriptor=635, MaxFileDescriptor=60000, SystemLoadAverage=144, ProcessCount=170, AvailableMemoryMB=7921 2023-03-09 01:56:13,877 DEBUG [Listener at localhost/33973] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-09 01:56:13,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xEA\xA9)\x84C\x00y-?\xBF=y\x17%6\xFE"\x10J\xDC\x99\xBAP\x90\x8Cc\x80\x1D\xA4\xBD(\x0F\xBD|*\x10\xDBh*\xD7\x89\xD0\x0B\xFA\xED\x02\xAEk\x9A\x1F\xFD\x092\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-09 01:56:13,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-09 01:56:13,935 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-09 01:56:13,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-09 01:56:13,936 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-09 01:56:13,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-09 01:56:13,938 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:13,938 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:13,940 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b empty. 2023-03-09 01:56:13,940 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:13,940 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764 empty. 2023-03-09 01:56:13,940 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a empty. 2023-03-09 01:56:13,940 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:13,940 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:13,940 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:13,940 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-09 01:56:13,947 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-09 01:56:13,948 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 1d7085c097705f5d7ab55f65b311318a, NAME => 'testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xEA\xA9)\x84C\x00y-?\xBF=y\x17%6\xFE"\x10J\xDC\x99\xBAP\x90\x8Cc\x80\x1D\xA4\xBD(\x0F\xBD|*\x10\xDBh*\xD7\x89\xD0\x0B\xFA\xED\x02\xAEk\x9A\x1F\xFD\x092\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:56:13,948 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 88ce0b923b48d5ff3b9b8acd71a12764, NAME => 'testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xEA\xA9)\x84C\x00y-?\xBF=y\x17%6\xFE"\x10J\xDC\x99\xBAP\x90\x8Cc\x80\x1D\xA4\xBD(\x0F\xBD|*\x10\xDBh*\xD7\x89\xD0\x0B\xFA\xED\x02\xAEk\x9A\x1F\xFD\x092\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:56:13,948 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4f486e10a1fba608022f8d64d072d40b, NAME => 'testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xEA\xA9)\x84C\x00y-?\xBF=y\x17%6\xFE"\x10J\xDC\x99\xBAP\x90\x8Cc\x80\x1D\xA4\xBD(\x0F\xBD|*\x10\xDBh*\xD7\x89\xD0\x0B\xFA\xED\x02\xAEk\x9A\x1F\xFD\x092\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/.tmp 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 4f486e10a1fba608022f8d64d072d40b, disabling compactions & flushes 2023-03-09 01:56:13,965 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. after waiting 0 ms 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:13,965 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 88ce0b923b48d5ff3b9b8acd71a12764, disabling compactions & flushes 2023-03-09 01:56:13,965 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. after waiting 0 ms 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:13,965 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:13,965 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:14,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-09 01:56:14,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-09 01:56:14,365 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:56:14,365 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 1d7085c097705f5d7ab55f65b311318a, disabling compactions & flushes 2023-03-09 01:56:14,365 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:14,365 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:14,365 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. after waiting 0 ms 2023-03-09 01:56:14,365 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:14,365 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:14,365 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:14,367 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-09 01:56:14,368 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326974368"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326974368"}]},"ts":"1678326974368"} 2023-03-09 01:56:14,368 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678326974368"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326974368"}]},"ts":"1678326974368"} 2023-03-09 01:56:14,368 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326974368"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1678326974368"}]},"ts":"1678326974368"} 2023-03-09 01:56:14,370 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-09 01:56:14,371 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-09 01:56:14,371 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326974371"}]},"ts":"1678326974371"} 2023-03-09 01:56:14,372 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-09 01:56:14,374 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=4f486e10a1fba608022f8d64d072d40b, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=88ce0b923b48d5ff3b9b8acd71a12764, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1d7085c097705f5d7ab55f65b311318a, ASSIGN}] 2023-03-09 01:56:14,376 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1d7085c097705f5d7ab55f65b311318a, ASSIGN 2023-03-09 01:56:14,376 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=88ce0b923b48d5ff3b9b8acd71a12764, ASSIGN 2023-03-09 01:56:14,376 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=4f486e10a1fba608022f8d64d072d40b, ASSIGN 2023-03-09 01:56:14,376 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1d7085c097705f5d7ab55f65b311318a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:56:14,376 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=88ce0b923b48d5ff3b9b8acd71a12764, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:56:14,376 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=4f486e10a1fba608022f8d64d072d40b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46511,1678326819757; forceNewPlan=false, retain=false 2023-03-09 01:56:14,528 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=4f486e10a1fba608022f8d64d072d40b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:14,528 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=88ce0b923b48d5ff3b9b8acd71a12764, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:14,528 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=1d7085c097705f5d7ab55f65b311318a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:14,529 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326974528"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326974528"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326974528"}]},"ts":"1678326974528"} 2023-03-09 01:56:14,529 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326974528"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326974528"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326974528"}]},"ts":"1678326974528"} 2023-03-09 01:56:14,529 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678326974528"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1678326974528"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1678326974528"}]},"ts":"1678326974528"} 2023-03-09 01:56:14,530 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure 4f486e10a1fba608022f8d64d072d40b, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:56:14,531 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=119, state=RUNNABLE; OpenRegionProcedure 1d7085c097705f5d7ab55f65b311318a, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:56:14,531 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure 88ce0b923b48d5ff3b9b8acd71a12764, server=jenkins-hbase4.apache.org,46511,1678326819757}] 2023-03-09 01:56:14,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-09 01:56:14,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:14,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1d7085c097705f5d7ab55f65b311318a, NAME => 'testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a.', STARTKEY => '2', ENDKEY => ''} 2023-03-09 01:56:14,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:14,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:56:14,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:14,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:14,707 INFO [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:14,708 DEBUG [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1 2023-03-09 01:56:14,708 DEBUG [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1 2023-03-09 01:56:14,708 INFO [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1d7085c097705f5d7ab55f65b311318a columnFamilyName family1 2023-03-09 01:56:14,709 INFO [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] regionserver.HStore(310): Store=1d7085c097705f5d7ab55f65b311318a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:56:14,710 INFO [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:14,710 DEBUG [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2 2023-03-09 01:56:14,710 DEBUG [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2 2023-03-09 01:56:14,711 INFO [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1d7085c097705f5d7ab55f65b311318a columnFamilyName family2 2023-03-09 01:56:14,712 INFO [StoreOpener-1d7085c097705f5d7ab55f65b311318a-1] regionserver.HStore(310): Store=1d7085c097705f5d7ab55f65b311318a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:56:14,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:14,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:14,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:56:14,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:14,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:56:14,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1d7085c097705f5d7ab55f65b311318a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59867106, jitterRate=-0.10791060328483582}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:56:14,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:14,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a., pid=121, masterSystemTime=1678326974681 2023-03-09 01:56:14,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:14,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:14,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:14,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4f486e10a1fba608022f8d64d072d40b, NAME => 'testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b.', STARTKEY => '', ENDKEY => '1'} 2023-03-09 01:56:14,719 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=1d7085c097705f5d7ab55f65b311318a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:14,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:14,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:56:14,719 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326974719"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326974719"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326974719"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326974719"}]},"ts":"1678326974719"} 2023-03-09 01:56:14,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:14,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:14,721 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-03-09 01:56:14,722 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; OpenRegionProcedure 1d7085c097705f5d7ab55f65b311318a, server=jenkins-hbase4.apache.org,46511,1678326819757 in 189 msec 2023-03-09 01:56:14,722 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1d7085c097705f5d7ab55f65b311318a, ASSIGN in 348 msec 2023-03-09 01:56:14,751 INFO [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:14,753 DEBUG [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1 2023-03-09 01:56:14,753 DEBUG [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1 2023-03-09 01:56:14,753 INFO [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4f486e10a1fba608022f8d64d072d40b columnFamilyName family1 2023-03-09 01:56:14,754 INFO [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] regionserver.HStore(310): Store=4f486e10a1fba608022f8d64d072d40b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:56:14,755 INFO [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:14,755 DEBUG [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2 2023-03-09 01:56:14,756 DEBUG [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2 2023-03-09 01:56:14,756 INFO [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4f486e10a1fba608022f8d64d072d40b columnFamilyName family2 2023-03-09 01:56:14,757 INFO [StoreOpener-4f486e10a1fba608022f8d64d072d40b-1] regionserver.HStore(310): Store=4f486e10a1fba608022f8d64d072d40b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:56:14,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:14,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:14,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:56:14,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:14,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:56:14,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4f486e10a1fba608022f8d64d072d40b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66117411, jitterRate=-0.01477380096912384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:56:14,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:14,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b., pid=120, masterSystemTime=1678326974681 2023-03-09 01:56:14,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:14,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:14,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:14,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 88ce0b923b48d5ff3b9b8acd71a12764, NAME => 'testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764.', STARTKEY => '1', ENDKEY => '2'} 2023-03-09 01:56:14,763 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=4f486e10a1fba608022f8d64d072d40b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:14,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:14,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-09 01:56:14,763 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1678326974763"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326974763"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326974763"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326974763"}]},"ts":"1678326974763"} 2023-03-09 01:56:14,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:14,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:14,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-03-09 01:56:14,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure 4f486e10a1fba608022f8d64d072d40b, server=jenkins-hbase4.apache.org,46511,1678326819757 in 234 msec 2023-03-09 01:56:14,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=4f486e10a1fba608022f8d64d072d40b, ASSIGN in 392 msec 2023-03-09 01:56:14,785 INFO [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:14,786 DEBUG [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1 2023-03-09 01:56:14,786 DEBUG [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1 2023-03-09 01:56:14,786 INFO [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88ce0b923b48d5ff3b9b8acd71a12764 columnFamilyName family1 2023-03-09 01:56:14,787 INFO [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] regionserver.HStore(310): Store=88ce0b923b48d5ff3b9b8acd71a12764/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:56:14,787 INFO [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:14,788 DEBUG [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2 2023-03-09 01:56:14,788 DEBUG [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2 2023-03-09 01:56:14,788 INFO [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88ce0b923b48d5ff3b9b8acd71a12764 columnFamilyName family2 2023-03-09 01:56:14,789 INFO [StoreOpener-88ce0b923b48d5ff3b9b8acd71a12764-1] regionserver.HStore(310): Store=88ce0b923b48d5ff3b9b8acd71a12764/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-09 01:56:14,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:14,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:14,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-09 01:56:14,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:14,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-09 01:56:14,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 88ce0b923b48d5ff3b9b8acd71a12764; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73363319, jitterRate=0.09319864213466644}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-09 01:56:14,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:14,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764., pid=122, masterSystemTime=1678326974681 2023-03-09 01:56:14,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:14,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:14,796 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=88ce0b923b48d5ff3b9b8acd71a12764, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:14,796 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1678326974796"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1678326974796"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1678326974796"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1678326974796"}]},"ts":"1678326974796"} 2023-03-09 01:56:14,798 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=118 2023-03-09 01:56:14,799 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure 88ce0b923b48d5ff3b9b8acd71a12764, server=jenkins-hbase4.apache.org,46511,1678326819757 in 266 msec 2023-03-09 01:56:14,800 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-03-09 01:56:14,800 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=88ce0b923b48d5ff3b9b8acd71a12764, ASSIGN in 425 msec 2023-03-09 01:56:14,800 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-09 01:56:14,800 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1678326974800"}]},"ts":"1678326974800"} 2023-03-09 01:56:14,801 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-09 01:56:14,803 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-09 01:56:14,804 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 870 msec 2023-03-09 01:56:15,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-09 01:56:15,040 INFO [Listener at localhost/33973] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-09 01:56:15,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:15,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:15,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:15,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:15,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-09 01:56:15,051 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-09 01:56:15,051 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:15,051 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-09 01:56:15,051 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-09 01:56:15,051 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:15,051 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:15,057 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:15,057 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:15,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:15,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:15,057 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:15,057 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:15,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:15,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:15,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:15,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:15,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-09 01:56:15,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-09 01:56:15,058 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-09 01:56:15,058 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:15,058 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-09 01:56:15,058 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:15,058 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:15,058 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:56:15,058 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. started... 2023-03-09 01:56:15,058 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:15,058 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:15,058 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. started... 2023-03-09 01:56:15,059 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1d7085c097705f5d7ab55f65b311318a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:56:15,059 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. started... 2023-03-09 01:56:15,060 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88ce0b923b48d5ff3b9b8acd71a12764 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:56:15,060 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f486e10a1fba608022f8d64d072d40b 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-09 01:56:15,104 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030923ce99a027df4259ad38f0221f032f16 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030923ce99a027df4259ad38f0221f032f16 2023-03-09 01:56:15,108 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309f2935db5f9054c7ca828f9a1c1bde52e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309f2935db5f9054c7ca828f9a1c1bde52e 2023-03-09 01:56:15,510 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303092a3dba32d2444c16b09bad208f5832d0 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303092a3dba32d2444c16b09bad208f5832d0 2023-03-09 01:56:15,519 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/ec6f4898541a47299c493a10f6e86e65 2023-03-09 01:56:15,522 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/9748477b8f50470c906c5dcb91472dff 2023-03-09 01:56:15,525 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/729723ef8f334666b4ca121702ddd544 2023-03-09 01:56:15,557 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309141de16da9ef4a1a881864681f868fba to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230309141de16da9ef4a1a881864681f868fba 2023-03-09 01:56:15,562 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/7199cd78b59b41b195dec9b3ad30db0e 2023-03-09 01:56:15,563 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309329ed69eae40442587a5b67f5f5959dd to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230309329ed69eae40442587a5b67f5f5959dd 2023-03-09 01:56:15,565 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/ec6f4898541a47299c493a10f6e86e65 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/ec6f4898541a47299c493a10f6e86e65 2023-03-09 01:56:15,589 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/ec6f4898541a47299c493a10f6e86e65, entries=4, sequenceid=5, filesize=5.5 K 2023-03-09 01:56:15,590 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/7199cd78b59b41b195dec9b3ad30db0e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/7199cd78b59b41b195dec9b3ad30db0e 2023-03-09 01:56:15,593 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/7199cd78b59b41b195dec9b3ad30db0e, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:56:15,593 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 88ce0b923b48d5ff3b9b8acd71a12764 in 533ms, sequenceid=5, compaction requested=false 2023-03-09 01:56:15,593 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-09 01:56:15,594 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:15,594 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:56:15,967 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/025ba107e42b4e1ba8234ddd5620090a 2023-03-09 01:56:15,968 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303090bd6f4ad5b5a445eaf677331067ca892 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303090bd6f4ad5b5a445eaf677331067ca892 2023-03-09 01:56:15,971 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/729723ef8f334666b4ca121702ddd544 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/729723ef8f334666b4ca121702ddd544 2023-03-09 01:56:15,995 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/729723ef8f334666b4ca121702ddd544, entries=4, sequenceid=5, filesize=5.5 K 2023-03-09 01:56:15,996 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/025ba107e42b4e1ba8234ddd5620090a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/025ba107e42b4e1ba8234ddd5620090a 2023-03-09 01:56:15,999 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/025ba107e42b4e1ba8234ddd5620090a, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:56:16,000 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 4f486e10a1fba608022f8d64d072d40b in 940ms, sequenceid=5, compaction requested=false 2023-03-09 01:56:16,001 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:16,001 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:16,001 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:56:16,373 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/1752228ae26f4fa49cbd4961580b0389 2023-03-09 01:56:16,377 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/9748477b8f50470c906c5dcb91472dff as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9748477b8f50470c906c5dcb91472dff 2023-03-09 01:56:16,400 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9748477b8f50470c906c5dcb91472dff, entries=4, sequenceid=5, filesize=5.5 K 2023-03-09 01:56:16,401 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/1752228ae26f4fa49cbd4961580b0389 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/1752228ae26f4fa49cbd4961580b0389 2023-03-09 01:56:16,404 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/1752228ae26f4fa49cbd4961580b0389, entries=2, sequenceid=5, filesize=5.1 K 2023-03-09 01:56:16,405 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 1d7085c097705f5d7ab55f65b311318a in 1346ms, sequenceid=5, compaction requested=false 2023-03-09 01:56:16,405 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:16,405 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:56:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:56:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-09 01:56:16,405 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-09 01:56:16,407 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,407 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:16,407 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:16,407 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:16,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-09 01:56:16,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12220b95[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:16,408 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-09 01:56:16,408 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:16,409 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:16,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:16,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:16,410 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,410 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:16,410 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-09 01:56:16,410 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-09 01:56:16,410 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:16,412 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,412 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-09 01:56:16,412 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:16,412 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-09 01:56:16,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:16,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:16,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-09 01:56:16,414 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-09 01:56:16,414 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:16,414 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:16,414 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-09 01:56:16,414 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:16,416 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,416 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:16,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,416 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:16,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:16,417 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:16,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:16,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,418 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:16,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:16,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:16,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:16,420 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:16,420 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-09 01:56:16,420 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:16,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:16,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:16,420 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:16,520 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:16,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:16,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:16,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:16,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:16,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:16,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:16,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-09 01:56:16,527 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-09 01:56:16,527 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:16,527 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-09 01:56:16,527 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-09 01:56:16,527 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,527 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:16,528 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:16,528 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:16,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:16,528 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:16,529 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:16,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:16,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:16,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:16,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:16,529 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-09 01:56:16,530 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-09 01:56:16,530 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-09 01:56:16,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-09 01:56:16,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:16,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-09 01:56:16,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:16,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. started... 2023-03-09 01:56:16,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:56:16,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:16,530 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88ce0b923b48d5ff3b9b8acd71a12764 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:56:16,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:16,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:16,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. started... 2023-03-09 01:56:16,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. started... 2023-03-09 01:56:16,530 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f486e10a1fba608022f8d64d072d40b 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:56:16,530 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1d7085c097705f5d7ab55f65b311318a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-09 01:56:16,574 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309b7f7ef2b4ed647e79a464c2d1f5e62d4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309b7f7ef2b4ed647e79a464c2d1f5e62d4 2023-03-09 01:56:16,973 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309a04673e1c3c144f19d7492d96df5f57a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309a04673e1c3c144f19d7492d96df5f57a 2023-03-09 01:56:16,980 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309cd1c54093e97402798492a00edf32ccc to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309cd1c54093e97402798492a00edf32ccc 2023-03-09 01:56:16,989 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/6cf851eca38f444c80bd1003415f3eba 2023-03-09 01:56:16,994 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/40dadacad13c45048850c3b98137efc7 2023-03-09 01:56:17,039 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023030998c2b7fabcae4ea9b9244f5e27410bbf to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023030998c2b7fabcae4ea9b9244f5e27410bbf 2023-03-09 01:56:17,388 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/b294eb8e742e4f30bc1708c6899cbbe1 2023-03-09 01:56:17,441 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309a4da9d7f067c45909681a8e99d2278b6 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230309a4da9d7f067c45909681a8e99d2278b6 2023-03-09 01:56:17,443 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/a4bf5e02b0c64e66beb3d3d83e586e30 2023-03-09 01:56:17,446 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/8e6940836c1a480394e09ea155fa482f 2023-03-09 01:56:17,447 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/6cf851eca38f444c80bd1003415f3eba as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/6cf851eca38f444c80bd1003415f3eba 2023-03-09 01:56:17,449 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/40dadacad13c45048850c3b98137efc7 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/40dadacad13c45048850c3b98137efc7 2023-03-09 01:56:17,471 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/6cf851eca38f444c80bd1003415f3eba, entries=4, sequenceid=9, filesize=5.5 K 2023-03-09 01:56:17,471 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/40dadacad13c45048850c3b98137efc7, entries=4, sequenceid=9, filesize=5.5 K 2023-03-09 01:56:17,472 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/a4bf5e02b0c64e66beb3d3d83e586e30 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/a4bf5e02b0c64e66beb3d3d83e586e30 2023-03-09 01:56:17,472 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/8e6940836c1a480394e09ea155fa482f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/8e6940836c1a480394e09ea155fa482f 2023-03-09 01:56:17,475 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/a4bf5e02b0c64e66beb3d3d83e586e30, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:56:17,475 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/8e6940836c1a480394e09ea155fa482f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:56:17,476 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 1d7085c097705f5d7ab55f65b311318a in 946ms, sequenceid=9, compaction requested=false 2023-03-09 01:56:17,476 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 88ce0b923b48d5ff3b9b8acd71a12764 in 946ms, sequenceid=9, compaction requested=false 2023-03-09 01:56:17,476 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:17,476 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:17,476 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:17,476 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:17,476 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:56:17,476 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:56:17,821 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309e26e7bdd2eb941eb8458eb1042956953 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230309e26e7bdd2eb941eb8458eb1042956953 2023-03-09 01:56:18,226 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/4aa7b8f6cc0342899da449a4c9c5e8cc 2023-03-09 01:56:18,230 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/b294eb8e742e4f30bc1708c6899cbbe1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/b294eb8e742e4f30bc1708c6899cbbe1 2023-03-09 01:56:18,254 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/b294eb8e742e4f30bc1708c6899cbbe1, entries=4, sequenceid=9, filesize=5.5 K 2023-03-09 01:56:18,255 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/4aa7b8f6cc0342899da449a4c9c5e8cc as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/4aa7b8f6cc0342899da449a4c9c5e8cc 2023-03-09 01:56:18,258 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/4aa7b8f6cc0342899da449a4c9c5e8cc, entries=2, sequenceid=9, filesize=5.1 K 2023-03-09 01:56:18,259 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 4f486e10a1fba608022f8d64d072d40b in 1729ms, sequenceid=9, compaction requested=false 2023-03-09 01:56:18,259 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:18,259 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:18,259 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:56:18,259 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:56:18,259 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,260 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-09 01:56:18,260 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-09 01:56:18,261 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,261 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:18,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:18,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:18,261 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:18,261 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:18,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-09 01:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@431aad35[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:18,262 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-09 01:56:18,263 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:18,265 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:18,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:18,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:18,265 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-09 01:56:18,265 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-09 01:56:18,265 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:18,265 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,265 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:18,266 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-09 01:56:18,266 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:18,266 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,266 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-09 01:56:18,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:18,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:18,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:18,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-09 01:56:18,268 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-09 01:56:18,268 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:18,268 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:18,268 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-09 01:56:18,269 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:18,270 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,270 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:18,270 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:18,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:18,270 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:18,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:18,272 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:18,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:18,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:18,274 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:18,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:18,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:18,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-09 01:56:18,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:18,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:18,274 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:18,375 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:18,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:18,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:18,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:18,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:18,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:18,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:18,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-09 01:56:18,381 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-09 01:56:18,381 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:18,381 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-09 01:56:18,381 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-09 01:56:18,381 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,381 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:18,384 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:18,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:18,384 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:18,386 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:18,386 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:18,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:18,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:18,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:18,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-09 01:56:18,394 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-09 01:56:18,394 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-09 01:56:18,394 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-09 01:56:18,394 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:18,394 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-09 01:56:18,395 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:18,395 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. started... 2023-03-09 01:56:18,395 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88ce0b923b48d5ff3b9b8acd71a12764 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:56:18,395 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:18,395 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. started... 2023-03-09 01:56:18,396 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1d7085c097705f5d7ab55f65b311318a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:56:18,399 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:56:18,399 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:18,399 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:18,399 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. started... 2023-03-09 01:56:18,400 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f486e10a1fba608022f8d64d072d40b 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-09 01:56:18,481 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303097eaa7909194845f88f44c56da3fc267a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303097eaa7909194845f88f44c56da3fc267a 2023-03-09 01:56:18,481 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309dff215db0a4f49e08c47401015aa8457 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309dff215db0a4f49e08c47401015aa8457 2023-03-09 01:56:18,486 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309360a93ec8b274022b9e3b2482e92f99c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309360a93ec8b274022b9e3b2482e92f99c 2023-03-09 01:56:18,500 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/fc6662cecb8346e9afa9feaccce84ab4 2023-03-09 01:56:18,500 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/a232a15f8e4745b9b5c1b44705c79768 2023-03-09 01:56:18,501 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/c5d7c014bc624a07a97de2028f00b994 2023-03-09 01:56:18,940 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303094eba421b0218432ebf91dbd508cdfc7c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303094eba421b0218432ebf91dbd508cdfc7c 2023-03-09 01:56:18,951 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309613da4cf55514fc79c8f8c267d35b26c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230309613da4cf55514fc79c8f8c267d35b26c 2023-03-09 01:56:18,951 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303095c4c29541b6f491894b0beaaa21c81e4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303095c4c29541b6f491894b0beaaa21c81e4 2023-03-09 01:56:19,348 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/16f4bebc198144f196b5056c8c0e620f 2023-03-09 01:56:19,353 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/fc6662cecb8346e9afa9feaccce84ab4 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/fc6662cecb8346e9afa9feaccce84ab4 2023-03-09 01:56:19,356 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/d74fd5a98e504820a1d0ba96ca6d7488 2023-03-09 01:56:19,356 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/be1d71608b3f47aa864099298a6938b7 2023-03-09 01:56:19,360 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/a232a15f8e4745b9b5c1b44705c79768 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/a232a15f8e4745b9b5c1b44705c79768 2023-03-09 01:56:19,360 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/c5d7c014bc624a07a97de2028f00b994 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/c5d7c014bc624a07a97de2028f00b994 2023-03-09 01:56:19,389 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/fc6662cecb8346e9afa9feaccce84ab4, entries=4, sequenceid=13, filesize=5.5 K 2023-03-09 01:56:19,390 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/16f4bebc198144f196b5056c8c0e620f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/16f4bebc198144f196b5056c8c0e620f 2023-03-09 01:56:19,394 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/a232a15f8e4745b9b5c1b44705c79768, entries=4, sequenceid=13, filesize=5.5 K 2023-03-09 01:56:19,394 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/16f4bebc198144f196b5056c8c0e620f, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:56:19,394 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/c5d7c014bc624a07a97de2028f00b994, entries=4, sequenceid=13, filesize=5.5 K 2023-03-09 01:56:19,394 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 1d7085c097705f5d7ab55f65b311318a in 999ms, sequenceid=13, compaction requested=true 2023-03-09 01:56:19,394 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:19,395 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:19,395 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:56:19,395 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/d74fd5a98e504820a1d0ba96ca6d7488 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/d74fd5a98e504820a1d0ba96ca6d7488 2023-03-09 01:56:19,395 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/be1d71608b3f47aa864099298a6938b7 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/be1d71608b3f47aa864099298a6938b7 2023-03-09 01:56:19,398 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/d74fd5a98e504820a1d0ba96ca6d7488, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:56:19,399 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/be1d71608b3f47aa864099298a6938b7, entries=2, sequenceid=13, filesize=5.1 K 2023-03-09 01:56:19,399 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 88ce0b923b48d5ff3b9b8acd71a12764 in 1004ms, sequenceid=13, compaction requested=true 2023-03-09 01:56:19,399 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:19,399 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 4f486e10a1fba608022f8d64d072d40b in 1000ms, sequenceid=13, compaction requested=true 2023-03-09 01:56:19,399 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:19,399 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:56:19,399 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:56:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:56:19,399 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,400 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-09 01:56:19,400 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-09 01:56:19,401 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:19,401 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:19,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:19,401 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:19,401 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:19,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-09 01:56:19,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5afa2bea[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:19,403 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-09 01:56:19,403 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:19,404 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:19,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:19,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:19,404 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-09 01:56:19,404 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-09 01:56:19,404 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:19,404 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,404 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:19,405 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,405 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-09 01:56:19,405 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:19,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,405 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-09 01:56:19,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:19,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:19,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:19,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:19,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:19,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:19,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:19,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:19,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-09 01:56:19,407 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-09 01:56:19,408 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:19,408 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:19,408 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-09 01:56:19,408 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:19,410 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,410 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:19,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,410 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:19,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:19,411 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:19,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:19,412 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:19,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:19,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:19,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-09 01:56:19,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:19,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:19,417 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:19,417 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:19,418 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:19,418 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:19,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:19,518 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:19,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:19,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:19,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:19,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-09 01:56:19,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:19,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:19,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-09 01:56:19,523 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-09 01:56:19,524 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:19,524 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-09 01:56:19,524 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-09 01:56:19,524 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,524 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:19,528 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,528 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:19,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:19,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:19,528 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:19,528 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:19,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:19,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:19,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:19,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:19,529 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-09 01:56:19,529 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-09 01:56:19,529 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-09 01:56:19,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-09 01:56:19,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:19,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-09 01:56:19,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:19,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:19,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. started... 2023-03-09 01:56:19,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. started... 2023-03-09 01:56:19,530 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1d7085c097705f5d7ab55f65b311318a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:56:19,530 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88ce0b923b48d5ff3b9b8acd71a12764 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:56:19,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:56:19,530 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:19,530 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:19,531 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. started... 2023-03-09 01:56:19,531 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f486e10a1fba608022f8d64d072d40b 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-09 01:56:19,584 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309b343730861c24dda958f79666b09f9d2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309b343730861c24dda958f79666b09f9d2 2023-03-09 01:56:19,972 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309104041a991e44f37b096e932d3809d95 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309104041a991e44f37b096e932d3809d95 2023-03-09 01:56:19,988 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030945a0ce862c0147eabdbc66628e136cec to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030945a0ce862c0147eabdbc66628e136cec 2023-03-09 01:56:19,988 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/9377da055759403899b7ea3f07993a1c 2023-03-09 01:56:19,999 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/2339f11a42224dda9de26d05afad2f52 2023-03-09 01:56:20,242 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-09 01:56:20,414 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/e69d866066fd427083d0819f34ccb04d 2023-03-09 01:56:20,431 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309c288c1ecf9c14436addc68aefbd1f6f9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230309c288c1ecf9c14436addc68aefbd1f6f9 2023-03-09 01:56:20,436 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/ba9f3c5a4e4c4ac58237ef41b83b0c1a 2023-03-09 01:56:20,437 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309751312727e2749f1bd6e71c263ee8188 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230309751312727e2749f1bd6e71c263ee8188 2023-03-09 01:56:20,442 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/9377da055759403899b7ea3f07993a1c as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9377da055759403899b7ea3f07993a1c 2023-03-09 01:56:20,467 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9377da055759403899b7ea3f07993a1c, entries=4, sequenceid=17, filesize=5.5 K 2023-03-09 01:56:20,468 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/ba9f3c5a4e4c4ac58237ef41b83b0c1a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/ba9f3c5a4e4c4ac58237ef41b83b0c1a 2023-03-09 01:56:20,471 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/ba9f3c5a4e4c4ac58237ef41b83b0c1a, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:56:20,472 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 1d7085c097705f5d7ab55f65b311318a in 942ms, sequenceid=17, compaction requested=true 2023-03-09 01:56:20,472 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:20,472 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:20,472 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:56:20,843 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/5d8a374192d24ab9a0e28e5dea5a9e45 2023-03-09 01:56:20,846 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/2339f11a42224dda9de26d05afad2f52 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/2339f11a42224dda9de26d05afad2f52 2023-03-09 01:56:20,853 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023030907caf52a679746deaa2505107ea2551e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023030907caf52a679746deaa2505107ea2551e 2023-03-09 01:56:20,871 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/2339f11a42224dda9de26d05afad2f52, entries=4, sequenceid=17, filesize=5.5 K 2023-03-09 01:56:20,872 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/5d8a374192d24ab9a0e28e5dea5a9e45 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/5d8a374192d24ab9a0e28e5dea5a9e45 2023-03-09 01:56:20,876 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/5d8a374192d24ab9a0e28e5dea5a9e45, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:56:20,876 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 88ce0b923b48d5ff3b9b8acd71a12764 in 1346ms, sequenceid=17, compaction requested=true 2023-03-09 01:56:20,876 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:20,877 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:20,877 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:56:21,267 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/52025e3fe6be48ffb0404410139b1612 2023-03-09 01:56:21,271 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/e69d866066fd427083d0819f34ccb04d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/e69d866066fd427083d0819f34ccb04d 2023-03-09 01:56:21,294 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/e69d866066fd427083d0819f34ccb04d, entries=4, sequenceid=17, filesize=5.5 K 2023-03-09 01:56:21,295 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/52025e3fe6be48ffb0404410139b1612 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/52025e3fe6be48ffb0404410139b1612 2023-03-09 01:56:21,298 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/52025e3fe6be48ffb0404410139b1612, entries=2, sequenceid=17, filesize=5.1 K 2023-03-09 01:56:21,299 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 4f486e10a1fba608022f8d64d072d40b in 1768ms, sequenceid=17, compaction requested=true 2023-03-09 01:56:21,299 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:21,299 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:21,299 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:56:21,299 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:56:21,299 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,299 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-09 01:56:21,299 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-09 01:56:21,302 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,302 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:21,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:21,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:21,302 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:21,302 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:21,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:21,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:21,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:21,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:21,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-09 01:56:21,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dfabc41[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:21,303 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-09 01:56:21,303 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:21,305 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:21,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:21,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:21,305 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-09 01:56:21,305 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,305 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:21,305 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-09 01:56:21,305 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:21,306 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,306 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-09 01:56:21,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:21,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:21,306 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:21,306 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-09 01:56:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:21,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-09 01:56:21,309 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-09 01:56:21,309 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:21,309 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:21,309 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-09 01:56:21,309 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:21,310 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,310 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:21,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,310 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:21,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:21,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:21,311 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:21,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:21,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,312 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:21,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:21,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,314 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,314 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:21,314 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:21,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:21,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:21,314 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:21,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-09 01:56:21,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:21,314 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:21,314 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:21,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:21,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:21,315 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,315 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:21,315 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:21,315 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:21,315 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,315 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:21,415 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:21,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:21,424 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:56:21,430 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:56:21,440 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:56:21,442 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:56:21,558 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-09 01:56:21,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-09 01:56:21,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-09 01:56:21,563 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-09 01:56:21,563 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:21,563 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-09 01:56:21,563 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-09 01:56:21,563 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,563 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:21,565 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,565 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:21,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:21,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:21,566 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:21,566 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-09 01:56:21,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:21,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:21,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-09 01:56:21,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:21,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-09 01:56:21,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-09 01:56:21,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-09 01:56:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-09 01:56:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-09 01:56:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-09 01:56:21,572 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:21,572 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:21,572 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:21,572 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. started... 2023-03-09 01:56:21,572 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. started... 2023-03-09 01:56:21,572 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. started... 2023-03-09 01:56:21,572 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1d7085c097705f5d7ab55f65b311318a 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:56:21,572 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-09 01:56:21,572 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 88ce0b923b48d5ff3b9b8acd71a12764 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:56:21,572 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-09 01:56:21,572 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 4f486e10a1fba608022f8d64d072d40b 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-09 01:56:22,415 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/e8bb6659f29b494ea98c260a3508fbcb 2023-03-09 01:56:22,415 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/af1f6e3ec0ba4639bd077b32c3fca6a2 2023-03-09 01:56:22,416 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/8e9a139b97d945dfb3c8c70cc6acc0f9 2023-03-09 01:56:22,457 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:56:22,457 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x39ea9093 to 127.0.0.1:50548 2023-03-09 01:56:22,457 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:22,459 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af1f6e3ec0ba4639bd077b32c3fca6a2 2023-03-09 01:56:22,459 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e8bb6659f29b494ea98c260a3508fbcb 2023-03-09 01:56:22,463 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e9a139b97d945dfb3c8c70cc6acc0f9 2023-03-09 01:56:22,879 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/b7d2d60519ae428c860c1e80dde38e59 2023-03-09 01:56:22,883 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/290453a3220740e587730b38a7299b2d 2023-03-09 01:56:22,883 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b7d2d60519ae428c860c1e80dde38e59 2023-03-09 01:56:22,884 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/13c9b6d930964656b49f7c6a2c970f69 2023-03-09 01:56:22,884 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/af1f6e3ec0ba4639bd077b32c3fca6a2 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/af1f6e3ec0ba4639bd077b32c3fca6a2 2023-03-09 01:56:22,886 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 290453a3220740e587730b38a7299b2d 2023-03-09 01:56:22,887 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/e8bb6659f29b494ea98c260a3508fbcb as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/e8bb6659f29b494ea98c260a3508fbcb 2023-03-09 01:56:22,887 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 13c9b6d930964656b49f7c6a2c970f69 2023-03-09 01:56:22,888 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/8e9a139b97d945dfb3c8c70cc6acc0f9 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/8e9a139b97d945dfb3c8c70cc6acc0f9 2023-03-09 01:56:22,917 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e9a139b97d945dfb3c8c70cc6acc0f9 2023-03-09 01:56:22,917 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/8e9a139b97d945dfb3c8c70cc6acc0f9, entries=3, sequenceid=23, filesize=5.2 K 2023-03-09 01:56:22,918 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/13c9b6d930964656b49f7c6a2c970f69 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/13c9b6d930964656b49f7c6a2c970f69 2023-03-09 01:56:22,919 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e8bb6659f29b494ea98c260a3508fbcb 2023-03-09 01:56:22,919 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/e8bb6659f29b494ea98c260a3508fbcb, entries=3, sequenceid=23, filesize=5.2 K 2023-03-09 01:56:22,920 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/290453a3220740e587730b38a7299b2d as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/290453a3220740e587730b38a7299b2d 2023-03-09 01:56:22,924 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 13c9b6d930964656b49f7c6a2c970f69 2023-03-09 01:56:22,924 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/13c9b6d930964656b49f7c6a2c970f69, entries=1, sequenceid=23, filesize=5.0 K 2023-03-09 01:56:22,925 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 4f486e10a1fba608022f8d64d072d40b in 1353ms, sequenceid=23, compaction requested=true 2023-03-09 01:56:22,925 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af1f6e3ec0ba4639bd077b32c3fca6a2 2023-03-09 01:56:22,925 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:22,925 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/af1f6e3ec0ba4639bd077b32c3fca6a2, entries=3, sequenceid=23, filesize=5.2 K 2023-03-09 01:56:22,925 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:22,926 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-09 01:56:22,926 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 290453a3220740e587730b38a7299b2d 2023-03-09 01:56:22,926 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/290453a3220740e587730b38a7299b2d, entries=1, sequenceid=23, filesize=5.0 K 2023-03-09 01:56:22,926 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/b7d2d60519ae428c860c1e80dde38e59 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/b7d2d60519ae428c860c1e80dde38e59 2023-03-09 01:56:22,927 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 88ce0b923b48d5ff3b9b8acd71a12764 in 1355ms, sequenceid=23, compaction requested=true 2023-03-09 01:56:22,927 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:22,927 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:22,927 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-09 01:56:22,929 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b7d2d60519ae428c860c1e80dde38e59 2023-03-09 01:56:22,930 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/b7d2d60519ae428c860c1e80dde38e59, entries=1, sequenceid=23, filesize=5.0 K 2023-03-09 01:56:22,930 INFO [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 1d7085c097705f5d7ab55f65b311318a in 1358ms, sequenceid=23, compaction requested=true 2023-03-09 01:56:22,930 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:22,930 DEBUG [rs(jenkins-hbase4.apache.org,46511,1678326819757)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:22,930 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-09 01:56:22,930 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-09 01:56:22,930 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,930 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-09 01:56:22,930 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-09 01:56:22,932 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,932 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:22,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:22,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:22,932 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:22,932 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-09 01:56:22,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:22,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:22,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:22,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:22,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46511,1678326819757' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-09 01:56:22,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f8666c8[Count = 0] remaining members to acquire global barrier 2023-03-09 01:56:22,934 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-09 01:56:22,934 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:22,935 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:22,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:22,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:22,935 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-09 01:56:22,936 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-09 01:56:22,936 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,46511,1678326819757' in zk 2023-03-09 01:56:22,936 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,936 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-09 01:56:22,937 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-09 01:56:22,937 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,937 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:22,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:22,937 DEBUG [member: 'jenkins-hbase4.apache.org,46511,1678326819757' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-09 01:56:22,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:22,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:22,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:22,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:22,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:22,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:22,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,46511,1678326819757': 2023-03-09 01:56:22,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46511,1678326819757' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-09 01:56:22,939 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-09 01:56:22,939 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-09 01:56:22,939 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-09 01:56:22,939 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-09 01:56:22,939 INFO [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-09 01:56:22,941 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:22,941 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:22,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:22,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:22,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-09 01:56:22,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:22,941 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:22,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-09 01:56:22,942 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:22,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:22,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-09 01:56:22,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:22,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:22,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-09 01:56:22,943 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:22,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-09 01:56:22,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-09 01:56:22,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:22,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-09 01:56:22,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-09 01:56:22,945 DEBUG [(jenkins-hbase4.apache.org,39575,1678326818523)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-09 01:56:22,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-09 01:56:22,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-09 01:56:22,945 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-09 01:56:22,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-09 01:56:22,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-09 01:56:22,946 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-09 01:56:22,946 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-09 01:56:22,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-09 01:56:23,046 DEBUG [Listener at localhost/33973] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-09 01:56:23,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39575] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-09 01:56:23,047 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:56:23,047 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 88ce0b923b48d5ff3b9b8acd71a12764/family1 is initiating major compaction (all files) 2023-03-09 01:56:23,047 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:56:23,047 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:56:23,048 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 88ce0b923b48d5ff3b9b8acd71a12764/family1 in testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:23,048 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/ec6f4898541a47299c493a10f6e86e65, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/40dadacad13c45048850c3b98137efc7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/a232a15f8e4745b9b5c1b44705c79768, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/2339f11a42224dda9de26d05afad2f52, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/e8bb6659f29b494ea98c260a3508fbcb] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp, totalSize=27.1 K 2023-03-09 01:56:23,048 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting ec6f4898541a47299c493a10f6e86e65, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326975047 2023-03-09 01:56:23,049 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 40dadacad13c45048850c3b98137efc7, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326976523 2023-03-09 01:56:23,049 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting a232a15f8e4745b9b5c1b44705c79768, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326978376 2023-03-09 01:56:23,049 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 2339f11a42224dda9de26d05afad2f52, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326979520 2023-03-09 01:56:23,049 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting e8bb6659f29b494ea98c260a3508fbcb, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:56:23,058 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 88ce0b923b48d5ff3b9b8acd71a12764#family1#compaction#197 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:56:23,495 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230309dbce3ac2a29f48dca87469e760102467_del 2023-03-09 01:56:23,495 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309dbce3ac2a29f48dca87469e760102467_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309dbce3ac2a29f48dca87469e760102467_del 2023-03-09 01:56:23,957 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06e55c3e9ddc46149381a16769451d2a 2023-03-09 01:56:23,957 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family1/06e55c3e9ddc46149381a16769451d2a as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/06e55c3e9ddc46149381a16769451d2a 2023-03-09 01:56:23,984 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06e55c3e9ddc46149381a16769451d2a 2023-03-09 01:56:23,985 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 88ce0b923b48d5ff3b9b8acd71a12764/family1 of 88ce0b923b48d5ff3b9b8acd71a12764 into 06e55c3e9ddc46149381a16769451d2a(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-09 01:56:23,985 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:23,985 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:56:23,985 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 88ce0b923b48d5ff3b9b8acd71a12764/family2 is initiating major compaction (all files) 2023-03-09 01:56:23,985 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:56:23,985 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:56:23,986 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 88ce0b923b48d5ff3b9b8acd71a12764/family2 in testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:23,986 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/7199cd78b59b41b195dec9b3ad30db0e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/8e6940836c1a480394e09ea155fa482f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/d74fd5a98e504820a1d0ba96ca6d7488, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/5d8a374192d24ab9a0e28e5dea5a9e45, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/290453a3220740e587730b38a7299b2d] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp, totalSize=25.4 K 2023-03-09 01:56:23,986 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 7199cd78b59b41b195dec9b3ad30db0e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326975047 2023-03-09 01:56:23,986 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 8e6940836c1a480394e09ea155fa482f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326976523 2023-03-09 01:56:23,987 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting d74fd5a98e504820a1d0ba96ca6d7488, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326978376 2023-03-09 01:56:23,987 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 5d8a374192d24ab9a0e28e5dea5a9e45, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326979520 2023-03-09 01:56:23,987 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 290453a3220740e587730b38a7299b2d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:56:23,995 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 88ce0b923b48d5ff3b9b8acd71a12764#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:56:24,053 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/ec6f4898541a47299c493a10f6e86e65, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/40dadacad13c45048850c3b98137efc7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/a232a15f8e4745b9b5c1b44705c79768, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/2339f11a42224dda9de26d05afad2f52, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/e8bb6659f29b494ea98c260a3508fbcb] to archive 2023-03-09 01:56:24,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:24,056 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/ec6f4898541a47299c493a10f6e86e65 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/ec6f4898541a47299c493a10f6e86e65 2023-03-09 01:56:24,057 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/40dadacad13c45048850c3b98137efc7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/40dadacad13c45048850c3b98137efc7 2023-03-09 01:56:24,058 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/a232a15f8e4745b9b5c1b44705c79768 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/a232a15f8e4745b9b5c1b44705c79768 2023-03-09 01:56:24,059 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/2339f11a42224dda9de26d05afad2f52 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/2339f11a42224dda9de26d05afad2f52 2023-03-09 01:56:24,060 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/e8bb6659f29b494ea98c260a3508fbcb to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/e8bb6659f29b494ea98c260a3508fbcb 2023-03-09 01:56:24,405 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230309de0d7a10abcf41b4b624f4ac97e8961f_del 2023-03-09 01:56:24,405 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230309de0d7a10abcf41b4b624f4ac97e8961f_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230309de0d7a10abcf41b4b624f4ac97e8961f_del 2023-03-09 01:56:25,218 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b13a5257f714f5ba633a3e72115eebc 2023-03-09 01:56:25,219 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/.tmp/family2/7b13a5257f714f5ba633a3e72115eebc as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/7b13a5257f714f5ba633a3e72115eebc 2023-03-09 01:56:25,223 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b13a5257f714f5ba633a3e72115eebc 2023-03-09 01:56:25,224 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 88ce0b923b48d5ff3b9b8acd71a12764/family2 of 88ce0b923b48d5ff3b9b8acd71a12764 into 7b13a5257f714f5ba633a3e72115eebc(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:56:25,224 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:25,224 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:56:25,224 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 1d7085c097705f5d7ab55f65b311318a/family1 is initiating major compaction (all files) 2023-03-09 01:56:25,224 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:56:25,224 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:56:25,225 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 1d7085c097705f5d7ab55f65b311318a/family1 in testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:25,225 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9748477b8f50470c906c5dcb91472dff, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/6cf851eca38f444c80bd1003415f3eba, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/fc6662cecb8346e9afa9feaccce84ab4, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9377da055759403899b7ea3f07993a1c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/af1f6e3ec0ba4639bd077b32c3fca6a2] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp, totalSize=27.1 K 2023-03-09 01:56:25,225 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 9748477b8f50470c906c5dcb91472dff, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326975047 2023-03-09 01:56:25,225 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 6cf851eca38f444c80bd1003415f3eba, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326976523 2023-03-09 01:56:25,226 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting fc6662cecb8346e9afa9feaccce84ab4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326978377 2023-03-09 01:56:25,226 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 9377da055759403899b7ea3f07993a1c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326979520 2023-03-09 01:56:25,226 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting af1f6e3ec0ba4639bd077b32c3fca6a2, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:56:25,234 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 1d7085c097705f5d7ab55f65b311318a#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:56:25,251 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/7199cd78b59b41b195dec9b3ad30db0e, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/8e6940836c1a480394e09ea155fa482f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/d74fd5a98e504820a1d0ba96ca6d7488, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/5d8a374192d24ab9a0e28e5dea5a9e45, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/290453a3220740e587730b38a7299b2d] to archive 2023-03-09 01:56:25,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:25,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/7199cd78b59b41b195dec9b3ad30db0e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/7199cd78b59b41b195dec9b3ad30db0e 2023-03-09 01:56:25,255 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/8e6940836c1a480394e09ea155fa482f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/8e6940836c1a480394e09ea155fa482f 2023-03-09 01:56:25,256 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/d74fd5a98e504820a1d0ba96ca6d7488 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/d74fd5a98e504820a1d0ba96ca6d7488 2023-03-09 01:56:25,257 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/5d8a374192d24ab9a0e28e5dea5a9e45 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/5d8a374192d24ab9a0e28e5dea5a9e45 2023-03-09 01:56:25,258 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/290453a3220740e587730b38a7299b2d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family2/290453a3220740e587730b38a7299b2d 2023-03-09 01:56:25,675 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230309631179035e744fd591faed98b175d2be_del 2023-03-09 01:56:25,675 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309631179035e744fd591faed98b175d2be_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309631179035e744fd591faed98b175d2be_del 2023-03-09 01:56:26,532 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f17782d7fe5941d38dc5716267624e38 2023-03-09 01:56:26,533 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family1/f17782d7fe5941d38dc5716267624e38 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/f17782d7fe5941d38dc5716267624e38 2023-03-09 01:56:26,567 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f17782d7fe5941d38dc5716267624e38 2023-03-09 01:56:26,568 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1d7085c097705f5d7ab55f65b311318a/family1 of 1d7085c097705f5d7ab55f65b311318a into f17782d7fe5941d38dc5716267624e38(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:56:26,568 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:26,568 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:56:26,568 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 1d7085c097705f5d7ab55f65b311318a/family2 is initiating major compaction (all files) 2023-03-09 01:56:26,568 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:56:26,568 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:56:26,568 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 1d7085c097705f5d7ab55f65b311318a/family2 in testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:26,569 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/1752228ae26f4fa49cbd4961580b0389, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/a4bf5e02b0c64e66beb3d3d83e586e30, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/16f4bebc198144f196b5056c8c0e620f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/ba9f3c5a4e4c4ac58237ef41b83b0c1a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/b7d2d60519ae428c860c1e80dde38e59] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp, totalSize=25.4 K 2023-03-09 01:56:26,569 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 1752228ae26f4fa49cbd4961580b0389, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326975047 2023-03-09 01:56:26,569 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting a4bf5e02b0c64e66beb3d3d83e586e30, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326976523 2023-03-09 01:56:26,570 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 16f4bebc198144f196b5056c8c0e620f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326978377 2023-03-09 01:56:26,570 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting ba9f3c5a4e4c4ac58237ef41b83b0c1a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326979520 2023-03-09 01:56:26,571 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting b7d2d60519ae428c860c1e80dde38e59, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:56:26,586 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 1d7085c097705f5d7ab55f65b311318a#family2#compaction#200 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:56:26,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9748477b8f50470c906c5dcb91472dff, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/6cf851eca38f444c80bd1003415f3eba, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/fc6662cecb8346e9afa9feaccce84ab4, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9377da055759403899b7ea3f07993a1c, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/af1f6e3ec0ba4639bd077b32c3fca6a2] to archive 2023-03-09 01:56:26,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:26,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9748477b8f50470c906c5dcb91472dff to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9748477b8f50470c906c5dcb91472dff 2023-03-09 01:56:26,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/6cf851eca38f444c80bd1003415f3eba to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/6cf851eca38f444c80bd1003415f3eba 2023-03-09 01:56:26,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/fc6662cecb8346e9afa9feaccce84ab4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/fc6662cecb8346e9afa9feaccce84ab4 2023-03-09 01:56:26,654 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9377da055759403899b7ea3f07993a1c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/9377da055759403899b7ea3f07993a1c 2023-03-09 01:56:26,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/af1f6e3ec0ba4639bd077b32c3fca6a2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/af1f6e3ec0ba4639bd077b32c3fca6a2 2023-03-09 01:56:26,997 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230309d4cb9e9d930d4960a52736697342c46e_del 2023-03-09 01:56:26,997 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309d4cb9e9d930d4960a52736697342c46e_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230309d4cb9e9d930d4960a52736697342c46e_del 2023-03-09 01:56:27,841 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc70623b30944f508916f7be7b4f719f 2023-03-09 01:56:27,842 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/.tmp/family2/dc70623b30944f508916f7be7b4f719f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/dc70623b30944f508916f7be7b4f719f 2023-03-09 01:56:27,845 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc70623b30944f508916f7be7b4f719f 2023-03-09 01:56:27,846 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1d7085c097705f5d7ab55f65b311318a/family2 of 1d7085c097705f5d7ab55f65b311318a into dc70623b30944f508916f7be7b4f719f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:56:27,846 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:27,846 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:56:27,846 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 4f486e10a1fba608022f8d64d072d40b/family1 is initiating major compaction (all files) 2023-03-09 01:56:27,846 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:56:27,846 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:56:27,847 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 4f486e10a1fba608022f8d64d072d40b/family1 in testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:27,847 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/729723ef8f334666b4ca121702ddd544, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/b294eb8e742e4f30bc1708c6899cbbe1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/c5d7c014bc624a07a97de2028f00b994, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/e69d866066fd427083d0819f34ccb04d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/8e9a139b97d945dfb3c8c70cc6acc0f9] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp, totalSize=27.1 K 2023-03-09 01:56:27,847 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 729723ef8f334666b4ca121702ddd544, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326975046 2023-03-09 01:56:27,847 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting b294eb8e742e4f30bc1708c6899cbbe1, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326976522 2023-03-09 01:56:27,848 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting c5d7c014bc624a07a97de2028f00b994, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326978376 2023-03-09 01:56:27,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/1752228ae26f4fa49cbd4961580b0389, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/a4bf5e02b0c64e66beb3d3d83e586e30, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/16f4bebc198144f196b5056c8c0e620f, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/ba9f3c5a4e4c4ac58237ef41b83b0c1a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/b7d2d60519ae428c860c1e80dde38e59] to archive 2023-03-09 01:56:27,849 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting e69d866066fd427083d0819f34ccb04d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326979520 2023-03-09 01:56:27,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:27,849 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 8e9a139b97d945dfb3c8c70cc6acc0f9, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:56:27,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/1752228ae26f4fa49cbd4961580b0389 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/1752228ae26f4fa49cbd4961580b0389 2023-03-09 01:56:27,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/a4bf5e02b0c64e66beb3d3d83e586e30 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/a4bf5e02b0c64e66beb3d3d83e586e30 2023-03-09 01:56:27,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/16f4bebc198144f196b5056c8c0e620f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/16f4bebc198144f196b5056c8c0e620f 2023-03-09 01:56:27,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/ba9f3c5a4e4c4ac58237ef41b83b0c1a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/ba9f3c5a4e4c4ac58237ef41b83b0c1a 2023-03-09 01:56:27,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/b7d2d60519ae428c860c1e80dde38e59 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family2/b7d2d60519ae428c860c1e80dde38e59 2023-03-09 01:56:27,859 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 4f486e10a1fba608022f8d64d072d40b#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:56:28,300 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309be25dd74e86f45b09ae066a68c6181c4_del 2023-03-09 01:56:28,301 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309be25dd74e86f45b09ae066a68c6181c4_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309be25dd74e86f45b09ae066a68c6181c4_del 2023-03-09 01:56:29,155 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 998b8ba080bd4c54bc1416f79245e9bd 2023-03-09 01:56:29,155 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family1/998b8ba080bd4c54bc1416f79245e9bd as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/998b8ba080bd4c54bc1416f79245e9bd 2023-03-09 01:56:29,177 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 998b8ba080bd4c54bc1416f79245e9bd 2023-03-09 01:56:29,178 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4f486e10a1fba608022f8d64d072d40b/family1 of 4f486e10a1fba608022f8d64d072d40b into 998b8ba080bd4c54bc1416f79245e9bd(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:56:29,178 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:29,178 DEBUG [Listener at localhost/33973] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-09 01:56:29,178 DEBUG [Listener at localhost/33973] regionserver.HStore(1912): 4f486e10a1fba608022f8d64d072d40b/family2 is initiating major compaction (all files) 2023-03-09 01:56:29,179 INFO [Listener at localhost/33973] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-09 01:56:29,179 INFO [Listener at localhost/33973] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-09 01:56:29,179 INFO [Listener at localhost/33973] regionserver.HRegion(2259): Starting compaction of 4f486e10a1fba608022f8d64d072d40b/family2 in testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:29,179 INFO [Listener at localhost/33973] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/025ba107e42b4e1ba8234ddd5620090a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/4aa7b8f6cc0342899da449a4c9c5e8cc, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/be1d71608b3f47aa864099298a6938b7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/52025e3fe6be48ffb0404410139b1612, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/13c9b6d930964656b49f7c6a2c970f69] into tmpdir=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp, totalSize=25.4 K 2023-03-09 01:56:29,179 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 025ba107e42b4e1ba8234ddd5620090a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1678326975046 2023-03-09 01:56:29,180 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 4aa7b8f6cc0342899da449a4c9c5e8cc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1678326976522 2023-03-09 01:56:29,180 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting be1d71608b3f47aa864099298a6938b7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1678326978376 2023-03-09 01:56:29,181 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 52025e3fe6be48ffb0404410139b1612, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1678326979520 2023-03-09 01:56:29,181 DEBUG [Listener at localhost/33973] compactions.Compactor(207): Compacting 13c9b6d930964656b49f7c6a2c970f69, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-09 01:56:29,188 INFO [Listener at localhost/33973] throttle.PressureAwareThroughputController(145): 4f486e10a1fba608022f8d64d072d40b#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-09 01:56:29,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/729723ef8f334666b4ca121702ddd544, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/b294eb8e742e4f30bc1708c6899cbbe1, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/c5d7c014bc624a07a97de2028f00b994, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/e69d866066fd427083d0819f34ccb04d, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/8e9a139b97d945dfb3c8c70cc6acc0f9] to archive 2023-03-09 01:56:29,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:29,251 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/729723ef8f334666b4ca121702ddd544 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/729723ef8f334666b4ca121702ddd544 2023-03-09 01:56:29,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/b294eb8e742e4f30bc1708c6899cbbe1 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/b294eb8e742e4f30bc1708c6899cbbe1 2023-03-09 01:56:29,253 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/c5d7c014bc624a07a97de2028f00b994 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/c5d7c014bc624a07a97de2028f00b994 2023-03-09 01:56:29,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/e69d866066fd427083d0819f34ccb04d to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/e69d866066fd427083d0819f34ccb04d 2023-03-09 01:56:29,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/8e9a139b97d945dfb3c8c70cc6acc0f9 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/8e9a139b97d945dfb3c8c70cc6acc0f9 2023-03-09 01:56:29,595 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309bd89f261b9584f7fb68a9185d7ff0386_del 2023-03-09 01:56:29,595 INFO [Listener at localhost/33973] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230309bd89f261b9584f7fb68a9185d7ff0386_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230309bd89f261b9584f7fb68a9185d7ff0386_del 2023-03-09 01:56:30,408 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fae01d469ef347ae8c79f1c9e12409ec 2023-03-09 01:56:30,409 DEBUG [Listener at localhost/33973] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/.tmp/family2/fae01d469ef347ae8c79f1c9e12409ec as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/fae01d469ef347ae8c79f1c9e12409ec 2023-03-09 01:56:30,412 INFO [Listener at localhost/33973] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fae01d469ef347ae8c79f1c9e12409ec 2023-03-09 01:56:30,413 INFO [Listener at localhost/33973] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4f486e10a1fba608022f8d64d072d40b/family2 of 4f486e10a1fba608022f8d64d072d40b into fae01d469ef347ae8c79f1c9e12409ec(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-09 01:56:30,413 DEBUG [Listener at localhost/33973] regionserver.HRegion(2289): Compaction status journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:30,416 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:56:30,419 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:56:30,422 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:56:30,423 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:56:30,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39575] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xEA\xA9)\x84C\x00y-?\xBF=y\x17%6\xFE"\x10J\xDC\x99\xBAP\x90\x8Cc\x80\x1D\xA4\xBD(\x0F\xBD|*\x10\xDBh*\xD7\x89\xD0\x0B\xFA\xED\x02\xAEk\x9A\x1F\xFD\x092\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-09 01:56:30,449 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-09 01:56:30,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/025ba107e42b4e1ba8234ddd5620090a, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/4aa7b8f6cc0342899da449a4c9c5e8cc, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/be1d71608b3f47aa864099298a6938b7, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/52025e3fe6be48ffb0404410139b1612, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/13c9b6d930964656b49f7c6a2c970f69] to archive 2023-03-09 01:56:30,449 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-09 01:56:30,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:30,450 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1678327590450 2023-03-09 01:56:30,450 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:56:30,451 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-09 01:56:30,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/025ba107e42b4e1ba8234ddd5620090a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/025ba107e42b4e1ba8234ddd5620090a 2023-03-09 01:56:30,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/4aa7b8f6cc0342899da449a4c9c5e8cc to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/4aa7b8f6cc0342899da449a4c9c5e8cc 2023-03-09 01:56:30,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/be1d71608b3f47aa864099298a6938b7 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/be1d71608b3f47aa864099298a6938b7 2023-03-09 01:56:30,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/52025e3fe6be48ffb0404410139b1612 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/52025e3fe6be48ffb0404410139b1612 2023-03-09 01:56:30,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/13c9b6d930964656b49f7c6a2c970f69 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family2/13c9b6d930964656b49f7c6a2c970f69 2023-03-09 01:56:30,781 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-09 01:56:30,803 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230309be25dd74e86f45b09ae066a68c6181c4_del 2023-03-09 01:56:30,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230309dbce3ac2a29f48dca87469e760102467_del 2023-03-09 01:56:30,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230309631179035e744fd591faed98b175d2be_del 2023-03-09 01:56:30,846 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-09 01:56:30,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] zookeeper.ReadOnlyZKClient(139): Connect 0x768bbbbf to 127.0.0.1:50548 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-09 01:56:30,850 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ec9400e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-09 01:56:30,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230309 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-09 01:56:31,311 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-09 01:56:31,803 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230309d0522af12c01474e81c9d27b85ce5949 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309d0522af12c01474e81c9d27b85ce5949 2023-03-09 01:56:31,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='testMajorCompactionFromAdmin' column='family1' 2023-03-09 01:56:31,816 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-09 01:56:31,818 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37826, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-09 01:56:31,827 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-09 01:56:31,828 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56592, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-09 01:56:31,860 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230309/family1/36919a85749f4a0db70a2c823e8e4572 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-09 01:56:31,861 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230309/family1/36919a85749f4a0db70a2c823e8e4572}] 2023-03-09 01:56:31,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230309/family1/36919a85749f4a0db70a2c823e8e4572 for inclusion in 1d7085c097705f5d7ab55f65b311318a/family1 2023-03-09 01:56:31,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-09 01:56:31,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-09 01:56:31,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:31,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230309/family1/36919a85749f4a0db70a2c823e8e4572 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__103h7upirhkapopqq4c14iaq3vtkq90ml3buta6ivk2ft00u5qccbo6ll9d79ipv/family1/36919a85749f4a0db70a2c823e8e4572 2023-03-09 01:56:31,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__103h7upirhkapopqq4c14iaq3vtkq90ml3buta6ivk2ft00u5qccbo6ll9d79ipv/family1/36919a85749f4a0db70a2c823e8e4572 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/8ed85d874ade41b79c6ac5aa311ef8ca_SeqId_28_ 2023-03-09 01:56:31,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__103h7upirhkapopqq4c14iaq3vtkq90ml3buta6ivk2ft00u5qccbo6ll9d79ipv/family1/36919a85749f4a0db70a2c823e8e4572 into 1d7085c097705f5d7ab55f65b311318a/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/8ed85d874ade41b79c6ac5aa311ef8ca_SeqId_28_ - updating store file list. 2023-03-09 01:56:31,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/8ed85d874ade41b79c6ac5aa311ef8ca_SeqId_28_ into 1d7085c097705f5d7ab55f65b311318a/family1 2023-03-09 01:56:31,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__103h7upirhkapopqq4c14iaq3vtkq90ml3buta6ivk2ft00u5qccbo6ll9d79ipv/family1/36919a85749f4a0db70a2c823e8e4572 into 1d7085c097705f5d7ab55f65b311318a/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/family1/8ed85d874ade41b79c6ac5aa311ef8ca_SeqId_28_) 2023-03-09 01:56:31,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__103h7upirhkapopqq4c14iaq3vtkq90ml3buta6ivk2ft00u5qccbo6ll9d79ipv/family1/36919a85749f4a0db70a2c823e8e4572 2023-03-09 01:56:31,928 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230309 table='testMajorCompactionFromAdmin' column='family1' 2023-03-09 01:56:31,929 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:31,930 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309104041a991e44f37b096e932d3809d95 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309104041a991e44f37b096e932d3809d95 2023-03-09 01:56:31,931 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309b7f7ef2b4ed647e79a464c2d1f5e62d4 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309b7f7ef2b4ed647e79a464c2d1f5e62d4 2023-03-09 01:56:31,932 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309dff215db0a4f49e08c47401015aa8457 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309dff215db0a4f49e08c47401015aa8457 2023-03-09 01:56:31,933 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309f2935db5f9054c7ca828f9a1c1bde52e to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309f2935db5f9054c7ca828f9a1c1bde52e 2023-03-09 01:56:31,933 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-09 01:56:31,933 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230309' 2023-03-09 01:56:31,933 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230309 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-09 01:56:31,933 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230309 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-09 01:56:32,114 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303090dd21d768b9a4b5cb09b8432f5c0c31f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303090dd21d768b9a4b5cb09b8432f5c0c31f 2023-03-09 01:56:32,115 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='testMajorCompactionFromAdmin' column='family1' 2023-03-09 01:56:32,169 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230309/family1/9a812fe232b24dea9b6c5f80a978ff9f first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-09 01:56:32,170 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230309/family1/9a812fe232b24dea9b6c5f80a978ff9f}] 2023-03-09 01:56:32,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230309/family1/9a812fe232b24dea9b6c5f80a978ff9f for inclusion in 4f486e10a1fba608022f8d64d072d40b/family1 2023-03-09 01:56:32,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-09 01:56:32,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first= last=1 2023-03-09 01:56:32,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:32,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230309/family1/9a812fe232b24dea9b6c5f80a978ff9f to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__4m16hpg0n3ochteouik12btn4hqdfslpkma4teop26e9u22eherq6mabv17ro9go/family1/9a812fe232b24dea9b6c5f80a978ff9f 2023-03-09 01:56:32,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__4m16hpg0n3ochteouik12btn4hqdfslpkma4teop26e9u22eherq6mabv17ro9go/family1/9a812fe232b24dea9b6c5f80a978ff9f as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/5755173a3b2b44738caf0daf87e986f2_SeqId_28_ 2023-03-09 01:56:32,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__4m16hpg0n3ochteouik12btn4hqdfslpkma4teop26e9u22eherq6mabv17ro9go/family1/9a812fe232b24dea9b6c5f80a978ff9f into 4f486e10a1fba608022f8d64d072d40b/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/5755173a3b2b44738caf0daf87e986f2_SeqId_28_ - updating store file list. 2023-03-09 01:56:32,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/5755173a3b2b44738caf0daf87e986f2_SeqId_28_ into 4f486e10a1fba608022f8d64d072d40b/family1 2023-03-09 01:56:32,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__4m16hpg0n3ochteouik12btn4hqdfslpkma4teop26e9u22eherq6mabv17ro9go/family1/9a812fe232b24dea9b6c5f80a978ff9f into 4f486e10a1fba608022f8d64d072d40b/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/family1/5755173a3b2b44738caf0daf87e986f2_SeqId_28_) 2023-03-09 01:56:32,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__4m16hpg0n3ochteouik12btn4hqdfslpkma4teop26e9u22eherq6mabv17ro9go/family1/9a812fe232b24dea9b6c5f80a978ff9f 2023-03-09 01:56:32,224 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230309 table='testMajorCompactionFromAdmin' column='family1' 2023-03-09 01:56:32,224 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:32,226 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303092a3dba32d2444c16b09bad208f5832d0 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303092a3dba32d2444c16b09bad208f5832d0 2023-03-09 01:56:32,227 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309360a93ec8b274022b9e3b2482e92f99c to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309360a93ec8b274022b9e3b2482e92f99c 2023-03-09 01:56:32,229 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030945a0ce862c0147eabdbc66628e136cec to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023030945a0ce862c0147eabdbc66628e136cec 2023-03-09 01:56:32,230 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309a04673e1c3c144f19d7492d96df5f57a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309a04673e1c3c144f19d7492d96df5f57a 2023-03-09 01:56:32,230 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-09 01:56:32,230 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230309' 2023-03-09 01:56:32,231 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230309 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-09 01:56:32,231 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230309 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-09 01:56:32,404 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023030965d735c3559c49abb7aaf3fd8c408710 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030965d735c3559c49abb7aaf3fd8c408710 2023-03-09 01:56:32,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230309 table='testMajorCompactionFromAdmin' column='family1' 2023-03-09 01:56:32,452 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230309/family1/0653becd09c843249037d9d30c8f8da3 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-09 01:56:32,453 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764., hostname=jenkins-hbase4.apache.org,46511,1678326819757, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230309/family1/0653becd09c843249037d9d30c8f8da3}] 2023-03-09 01:56:32,455 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(763): Validating hfile at hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230309/family1/0653becd09c843249037d9d30c8f8da3 for inclusion in 88ce0b923b48d5ff3b9b8acd71a12764/family1 2023-03-09 01:56:32,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-09 01:56:32,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-09 01:56:32,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegion(2446): Flush status journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:32,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230309/family1/0653becd09c843249037d9d30c8f8da3 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__1389i6gd0bagkjb9799kh30tinarldosfndvisnd9527o7gh3svbgn8ag8nr746d/family1/0653becd09c843249037d9d30c8f8da3 2023-03-09 01:56:32,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__1389i6gd0bagkjb9799kh30tinarldosfndvisnd9527o7gh3svbgn8ag8nr746d/family1/0653becd09c843249037d9d30c8f8da3 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/52a85d7200914dfe969caad6fc099f2c_SeqId_28_ 2023-03-09 01:56:32,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(849): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__1389i6gd0bagkjb9799kh30tinarldosfndvisnd9527o7gh3svbgn8ag8nr746d/family1/0653becd09c843249037d9d30c8f8da3 into 88ce0b923b48d5ff3b9b8acd71a12764/family1 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/52a85d7200914dfe969caad6fc099f2c_SeqId_28_ - updating store file list. 2023-03-09 01:56:32,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(882): Loaded HFile hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/52a85d7200914dfe969caad6fc099f2c_SeqId_28_ into 88ce0b923b48d5ff3b9b8acd71a12764/family1 2023-03-09 01:56:32,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.HStore(855): Successfully loaded hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__1389i6gd0bagkjb9799kh30tinarldosfndvisnd9527o7gh3svbgn8ag8nr746d/family1/0653becd09c843249037d9d30c8f8da3 into 88ce0b923b48d5ff3b9b8acd71a12764/family1 (new location: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/family1/52a85d7200914dfe969caad6fc099f2c_SeqId_28_) 2023-03-09 01:56:32,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46511] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/staging/jenkins__testMajorCompactionFromAdmin__1389i6gd0bagkjb9799kh30tinarldosfndvisnd9527o7gh3svbgn8ag8nr746d/family1/0653becd09c843249037d9d30c8f8da3 2023-03-09 01:56:32,503 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230309 table='testMajorCompactionFromAdmin' column='family1' 2023-03-09 01:56:32,504 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:32,505 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030923ce99a027df4259ad38f0221f032f16 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023030923ce99a027df4259ad38f0221f032f16 2023-03-09 01:56:32,506 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303097eaa7909194845f88f44c56da3fc267a to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303097eaa7909194845f88f44c56da3fc267a 2023-03-09 01:56:32,506 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309b343730861c24dda958f79666b09f9d2 to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309b343730861c24dda958f79666b09f9d2 2023-03-09 01:56:32,507 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309cd1c54093e97402798492a00edf32ccc to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309cd1c54093e97402798492a00edf32ccc 2023-03-09 01:56:32,507 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-09 01:56:32,507 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230309' 2023-03-09 01:56:32,508 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230309 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-09 01:56:32,508 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-09 01:56:32,508 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-09 01:56:32,508 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309be25dd74e86f45b09ae066a68c6181c4_del] 2023-03-09 01:56:32,508 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:32,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309be25dd74e86f45b09ae066a68c6181c4_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230309be25dd74e86f45b09ae066a68c6181c4_del 2023-03-09 01:56:32,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-09 01:56:32,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309dbce3ac2a29f48dca87469e760102467_del] 2023-03-09 01:56:32,510 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:32,511 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309dbce3ac2a29f48dca87469e760102467_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230309dbce3ac2a29f48dca87469e760102467_del 2023-03-09 01:56:32,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-09 01:56:32,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309631179035e744fd591faed98b175d2be_del] 2023-03-09 01:56:32,511 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] backup.HFileArchiver(360): Archiving compacted files. 2023-03-09 01:56:32,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309631179035e744fd591faed98b175d2be_del to hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230309631179035e744fd591faed98b175d2be_del 2023-03-09 01:56:32,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-09 01:56:32,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1678326823272] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-09 01:56:32,514 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-09 01:56:32,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.0660 sec 2023-03-09 01:56:32,529 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:56:32,531 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:56:32,533 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-09 01:56:32,534 DEBUG [Listener at localhost/33973] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-09 01:56:32,538 INFO [Listener at localhost/33973] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-09 01:56:32,553 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-09 01:56:32,582 INFO [Listener at localhost/33973] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 250), OpenFileDescriptor=641 (was 635) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=123 (was 144), ProcessCount=170 (was 170), AvailableMemoryMB=7723 (was 7921) 2023-03-09 01:56:32,582 DEBUG [Listener at localhost/33973] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x143ae868 to 127.0.0.1:50548 2023-03-09 01:56:32,582 DEBUG [Listener at localhost/33973] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:32,583 INFO [Listener at localhost/33973] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-09 01:56:32,583 INFO [Listener at localhost/33973] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-03-09 01:56:32,583 DEBUG [Listener at localhost/33973] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x449b853a to 127.0.0.1:50548 2023-03-09 01:56:32,583 DEBUG [Listener at localhost/33973] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:32,583 DEBUG [Listener at localhost/33973] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-09 01:56:32,583 DEBUG [Listener at localhost/33973] util.JVMClusterUtil(257): Found active master hash=76632293, stopped=false 2023-03-09 01:56:32,583 INFO [Listener at localhost/33973] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:56:32,585 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-09 01:56:32,585 INFO [Listener at localhost/33973] procedure2.ProcedureExecutor(629): Stopping 2023-03-09 01:56:32,585 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:56:32,585 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-09 01:56:32,585 DEBUG [Listener at localhost/33973] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33053f4c to 127.0.0.1:50548 2023-03-09 01:56:32,585 DEBUG [Listener at localhost/33973] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:32,585 INFO [Listener at localhost/33973] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,46511,1678326819757' ***** 2023-03-09 01:56:32,585 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-09 01:56:32,585 INFO [Listener at localhost/33973] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-09 01:56:32,585 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-09 01:56:32,586 INFO [RS:0;jenkins-hbase4:46511] regionserver.HeapMemoryManager(220): Stopping 2023-03-09 01:56:32,586 INFO [RS:0;jenkins-hbase4:46511] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-09 01:56:32,586 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-09 01:56:32,586 INFO [RS:0;jenkins-hbase4:46511] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-09 01:56:32,586 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for abbc0d228ef2735615e5e8f7e1c367fc 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for eb9b94d3d4712803b80d37dce323940c 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for f0e8e486b536a2c5f3a4c2d719d586ce 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 18e9a9ff9da4b5322903ba24ba13b2e1 2023-03-09 01:56:32,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing abbc0d228ef2735615e5e8f7e1c367fc, disabling compactions & flushes 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 3a92a0ebc17a6fe227772466246c6162 2023-03-09 01:56:32,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 527ef65d68bd3fd8127836853154b8b0 2023-03-09 01:56:32,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 88ce0b923b48d5ff3b9b8acd71a12764 2023-03-09 01:56:32,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. after waiting 0 ms 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 00f768e6a69add6056570cdcb3ad746e 2023-03-09 01:56:32,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 2c8c3d2b24a65eed3f072e8124247764 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for fd53913779b3fcab65c638e819ce5953 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for cd5a60b17949509277f0e081b9d03fe0 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 80524e4cfd7024a7ff50a654d546b0de 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 51e7af8c834c7e1949ec8753f22a28d7 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 1d7085c097705f5d7ab55f65b311318a 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for b0d3ba46f6423b83104f4f0f94be0362 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 4f486e10a1fba608022f8d64d072d40b 2023-03-09 01:56:32,587 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 519ef0416144e355c478496bd9b26e48 2023-03-09 01:56:32,588 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 0ea7b302e0cce81de94432b7e794c655 2023-03-09 01:56:32,588 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 11ddcd7bf8df7d1e6665f6c9cc6acd73 2023-03-09 01:56:32,588 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for a4a8be322482491239e225079ce61b3a 2023-03-09 01:56:32,588 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:32,588 DEBUG [RS:0;jenkins-hbase4:46511] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x177d308c to 127.0.0.1:50548 2023-03-09 01:56:32,588 DEBUG [RS:0;jenkins-hbase4:46511] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:32,588 INFO [RS:0;jenkins-hbase4:46511] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-09 01:56:32,588 INFO [RS:0;jenkins-hbase4:46511] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-09 01:56:32,588 INFO [RS:0;jenkins-hbase4:46511] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-09 01:56:32,588 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-09 01:56:32,590 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-09 01:56:32,590 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-09 01:56:32,590 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-09 01:56:32,590 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-09 01:56:32,590 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-09 01:56:32,590 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-09 01:56:32,590 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-09 01:56:32,591 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1505): Waiting on 00f768e6a69add6056570cdcb3ad746e, 0ea7b302e0cce81de94432b7e794c655, 11ddcd7bf8df7d1e6665f6c9cc6acd73, 1588230740, 18e9a9ff9da4b5322903ba24ba13b2e1, 1d7085c097705f5d7ab55f65b311318a, 2c8c3d2b24a65eed3f072e8124247764, 3a92a0ebc17a6fe227772466246c6162, 4f486e10a1fba608022f8d64d072d40b, 519ef0416144e355c478496bd9b26e48, 51e7af8c834c7e1949ec8753f22a28d7, 527ef65d68bd3fd8127836853154b8b0, 80524e4cfd7024a7ff50a654d546b0de, 88ce0b923b48d5ff3b9b8acd71a12764, a4a8be322482491239e225079ce61b3a, abbc0d228ef2735615e5e8f7e1c367fc, b0d3ba46f6423b83104f4f0f94be0362, cd5a60b17949509277f0e081b9d03fe0, eb9b94d3d4712803b80d37dce323940c, f0e8e486b536a2c5f3a4c2d719d586ce, fd53913779b3fcab65c638e819ce5953 2023-03-09 01:56:32,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/abbc0d228ef2735615e5e8f7e1c367fc/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-09 01:56:32,592 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:56:32,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for abbc0d228ef2735615e5e8f7e1c367fc: 2023-03-09 01:56:32,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1678326914692.abbc0d228ef2735615e5e8f7e1c367fc. 2023-03-09 01:56:32,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eb9b94d3d4712803b80d37dce323940c, disabling compactions & flushes 2023-03-09 01:56:32,592 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:56:32,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:56:32,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. after waiting 0 ms 2023-03-09 01:56:32,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:56:32,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/eb9b94d3d4712803b80d37dce323940c/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-09 01:56:32,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:56:32,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eb9b94d3d4712803b80d37dce323940c: 2023-03-09 01:56:32,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1678326859497.eb9b94d3d4712803b80d37dce323940c. 2023-03-09 01:56:32,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f0e8e486b536a2c5f3a4c2d719d586ce, disabling compactions & flushes 2023-03-09 01:56:32,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:32,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:32,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. after waiting 0 ms 2023-03-09 01:56:32,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:32,599 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithMonthlyPolicy/f0e8e486b536a2c5f3a4c2d719d586ce/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-09 01:56:32,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:32,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f0e8e486b536a2c5f3a4c2d719d586ce: 2023-03-09 01:56:32,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1678326960075.f0e8e486b536a2c5f3a4c2d719d586ce. 2023-03-09 01:56:32,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 18e9a9ff9da4b5322903ba24ba13b2e1, disabling compactions & flushes 2023-03-09 01:56:32,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:56:32,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:56:32,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. after waiting 0 ms 2023-03-09 01:56:32,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:56:32,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/18e9a9ff9da4b5322903ba24ba13b2e1/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-09 01:56:32,603 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-09 01:56:32,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:56:32,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 18e9a9ff9da4b5322903ba24ba13b2e1: 2023-03-09 01:56:32,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1678326914692.18e9a9ff9da4b5322903ba24ba13b2e1. 2023-03-09 01:56:32,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3a92a0ebc17a6fe227772466246c6162, disabling compactions & flushes 2023-03-09 01:56:32,604 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:56:32,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:56:32,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. after waiting 0 ms 2023-03-09 01:56:32,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:56:32,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/3a92a0ebc17a6fe227772466246c6162/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-09 01:56:32,607 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:56:32,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3a92a0ebc17a6fe227772466246c6162: 2023-03-09 01:56:32,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1678326876037.3a92a0ebc17a6fe227772466246c6162. 2023-03-09 01:56:32,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 527ef65d68bd3fd8127836853154b8b0, disabling compactions & flushes 2023-03-09 01:56:32,607 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:56:32,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:56:32,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. after waiting 0 ms 2023-03-09 01:56:32,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:56:32,609 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/527ef65d68bd3fd8127836853154b8b0/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-09 01:56:32,610 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:56:32,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 527ef65d68bd3fd8127836853154b8b0: 2023-03-09 01:56:32,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1678326891794.527ef65d68bd3fd8127836853154b8b0. 2023-03-09 01:56:32,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 88ce0b923b48d5ff3b9b8acd71a12764, disabling compactions & flushes 2023-03-09 01:56:32,610 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:32,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:32,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. after waiting 0 ms 2023-03-09 01:56:32,610 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:32,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/88ce0b923b48d5ff3b9b8acd71a12764/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-09 01:56:32,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:32,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 88ce0b923b48d5ff3b9b8acd71a12764: 2023-03-09 01:56:32,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1678326973901.88ce0b923b48d5ff3b9b8acd71a12764. 2023-03-09 01:56:32,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 00f768e6a69add6056570cdcb3ad746e, disabling compactions & flushes 2023-03-09 01:56:32,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:56:32,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:56:32,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. after waiting 0 ms 2023-03-09 01:56:32,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:56:32,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testCompactionWithHFileLink/00f768e6a69add6056570cdcb3ad746e/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-09 01:56:32,619 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:56:32,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 00f768e6a69add6056570cdcb3ad746e: 2023-03-09 01:56:32,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1678326914692.00f768e6a69add6056570cdcb3ad746e. 2023-03-09 01:56:32,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2c8c3d2b24a65eed3f072e8124247764, disabling compactions & flushes 2023-03-09 01:56:32,619 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:56:32,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:56:32,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. after waiting 0 ms 2023-03-09 01:56:32,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:56:32,621 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMinorCompactionWithWeeklyPolicy/2c8c3d2b24a65eed3f072e8124247764/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-09 01:56:32,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:56:32,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2c8c3d2b24a65eed3f072e8124247764: 2023-03-09 01:56:32,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1678326847624.2c8c3d2b24a65eed3f072e8124247764. 2023-03-09 01:56:32,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fd53913779b3fcab65c638e819ce5953, disabling compactions & flushes 2023-03-09 01:56:32,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:56:32,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:56:32,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. after waiting 0 ms 2023-03-09 01:56:32,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:56:32,625 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/fd53913779b3fcab65c638e819ce5953/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-09 01:56:32,626 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:56:32,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fd53913779b3fcab65c638e819ce5953: 2023-03-09 01:56:32,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1678326940945.fd53913779b3fcab65c638e819ce5953. 2023-03-09 01:56:32,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cd5a60b17949509277f0e081b9d03fe0, disabling compactions & flushes 2023-03-09 01:56:32,626 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:56:32,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:56:32,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. after waiting 0 ms 2023-03-09 01:56:32,626 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:56:32,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/cd5a60b17949509277f0e081b9d03fe0/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-09 01:56:32,629 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:56:32,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cd5a60b17949509277f0e081b9d03fe0: 2023-03-09 01:56:32,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1678326859497.cd5a60b17949509277f0e081b9d03fe0. 2023-03-09 01:56:32,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 80524e4cfd7024a7ff50a654d546b0de, disabling compactions & flushes 2023-03-09 01:56:32,629 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:56:32,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:56:32,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. after waiting 0 ms 2023-03-09 01:56:32,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:56:32,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/80524e4cfd7024a7ff50a654d546b0de/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-09 01:56:32,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:56:32,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 80524e4cfd7024a7ff50a654d546b0de: 2023-03-09 01:56:32,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1678326823407.80524e4cfd7024a7ff50a654d546b0de. 2023-03-09 01:56:32,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 51e7af8c834c7e1949ec8753f22a28d7, disabling compactions & flushes 2023-03-09 01:56:32,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:56:32,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:56:32,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. after waiting 0 ms 2023-03-09 01:56:32,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:56:32,635 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testScannerOnBulkLoadRefHFiles/51e7af8c834c7e1949ec8753f22a28d7/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-09 01:56:32,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:56:32,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 51e7af8c834c7e1949ec8753f22a28d7: 2023-03-09 01:56:32,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1678326859497.51e7af8c834c7e1949ec8753f22a28d7. 2023-03-09 01:56:32,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1d7085c097705f5d7ab55f65b311318a, disabling compactions & flushes 2023-03-09 01:56:32,637 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:32,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:32,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. after waiting 0 ms 2023-03-09 01:56:32,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:32,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/1d7085c097705f5d7ab55f65b311318a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-09 01:56:32,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:32,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1d7085c097705f5d7ab55f65b311318a: 2023-03-09 01:56:32,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1678326973901.1d7085c097705f5d7ab55f65b311318a. 2023-03-09 01:56:32,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b0d3ba46f6423b83104f4f0f94be0362, disabling compactions & flushes 2023-03-09 01:56:32,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:56:32,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:56:32,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. after waiting 0 ms 2023-03-09 01:56:32,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:56:32,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/b0d3ba46f6423b83104f4f0f94be0362/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-09 01:56:32,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:56:32,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b0d3ba46f6423b83104f4f0f94be0362: 2023-03-09 01:56:32,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1678326940945.b0d3ba46f6423b83104f4f0f94be0362. 2023-03-09 01:56:32,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4f486e10a1fba608022f8d64d072d40b, disabling compactions & flushes 2023-03-09 01:56:32,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:32,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:32,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. after waiting 0 ms 2023-03-09 01:56:32,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:32,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionFromAdmin/4f486e10a1fba608022f8d64d072d40b/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-09 01:56:32,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:32,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4f486e10a1fba608022f8d64d072d40b: 2023-03-09 01:56:32,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1678326973901.4f486e10a1fba608022f8d64d072d40b. 2023-03-09 01:56:32,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 519ef0416144e355c478496bd9b26e48, disabling compactions & flushes 2023-03-09 01:56:32,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:56:32,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:56:32,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. after waiting 0 ms 2023-03-09 01:56:32,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:56:32,648 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-09 01:56:32,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithMonthlyPolicy/519ef0416144e355c478496bd9b26e48/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-09 01:56:32,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:56:32,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 519ef0416144e355c478496bd9b26e48: 2023-03-09 01:56:32,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1678326837254.519ef0416144e355c478496bd9b26e48. 2023-03-09 01:56:32,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0ea7b302e0cce81de94432b7e794c655, disabling compactions & flushes 2023-03-09 01:56:32,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:56:32,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:56:32,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. after waiting 0 ms 2023-03-09 01:56:32,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:56:32,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 0ea7b302e0cce81de94432b7e794c655 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-09 01:56:32,791 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1505): Waiting on 0ea7b302e0cce81de94432b7e794c655, 11ddcd7bf8df7d1e6665f6c9cc6acd73, 1588230740, a4a8be322482491239e225079ce61b3a 2023-03-09 01:56:32,991 DEBUG [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1505): Waiting on 0ea7b302e0cce81de94432b7e794c655, 11ddcd7bf8df7d1e6665f6c9cc6acd73, 1588230740, a4a8be322482491239e225079ce61b3a 2023-03-09 01:56:33,000 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/.tmp/info/cdf97cbfd4564a92b76a818550671c38 2023-03-09 01:56:33,006 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cdf97cbfd4564a92b76a818550671c38 2023-03-09 01:56:33,016 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/.tmp/rep_barrier/679a5cda20984f7ebb6db40348083b5e 2023-03-09 01:56:33,020 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 679a5cda20984f7ebb6db40348083b5e 2023-03-09 01:56:33,026 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/.tmp/table/684ca4dc4b8f452eb0ce583cd7415f12 2023-03-09 01:56:33,029 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 684ca4dc4b8f452eb0ce583cd7415f12 2023-03-09 01:56:33,030 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/.tmp/info/cdf97cbfd4564a92b76a818550671c38 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/info/cdf97cbfd4564a92b76a818550671c38 2023-03-09 01:56:33,033 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cdf97cbfd4564a92b76a818550671c38 2023-03-09 01:56:33,033 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/info/cdf97cbfd4564a92b76a818550671c38, entries=226, sequenceid=130, filesize=37.9 K 2023-03-09 01:56:33,034 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/.tmp/rep_barrier/679a5cda20984f7ebb6db40348083b5e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/rep_barrier/679a5cda20984f7ebb6db40348083b5e 2023-03-09 01:56:33,038 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 679a5cda20984f7ebb6db40348083b5e 2023-03-09 01:56:33,038 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/rep_barrier/679a5cda20984f7ebb6db40348083b5e, entries=1, sequenceid=130, filesize=5.0 K 2023-03-09 01:56:33,039 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/.tmp/table/684ca4dc4b8f452eb0ce583cd7415f12 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/table/684ca4dc4b8f452eb0ce583cd7415f12 2023-03-09 01:56:33,042 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 684ca4dc4b8f452eb0ce583cd7415f12 2023-03-09 01:56:33,042 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/table/684ca4dc4b8f452eb0ce583cd7415f12, entries=27, sequenceid=130, filesize=6.6 K 2023-03-09 01:56:33,043 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 453ms, sequenceid=130, compaction requested=false 2023-03-09 01:56:33,054 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-09 01:56:33,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655/.tmp/info/665e85fe072f47a2ad01e65a30085291 2023-03-09 01:56:33,055 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-09 01:56:33,055 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-09 01:56:33,055 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-09 01:56:33,055 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-09 01:56:33,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655/.tmp/info/665e85fe072f47a2ad01e65a30085291 as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655/info/665e85fe072f47a2ad01e65a30085291 2023-03-09 01:56:33,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655/info/665e85fe072f47a2ad01e65a30085291, entries=3, sequenceid=7, filesize=4.8 K 2023-03-09 01:56:33,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 0ea7b302e0cce81de94432b7e794c655 in 415ms, sequenceid=7, compaction requested=false 2023-03-09 01:56:33,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/hbase/namespace/0ea7b302e0cce81de94432b7e794c655/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-09 01:56:33,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:56:33,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0ea7b302e0cce81de94432b7e794c655: 2023-03-09 01:56:33,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1678326822139.0ea7b302e0cce81de94432b7e794c655. 2023-03-09 01:56:33,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 11ddcd7bf8df7d1e6665f6c9cc6acd73, disabling compactions & flushes 2023-03-09 01:56:33,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:56:33,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:56:33,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. after waiting 0 ms 2023-03-09 01:56:33,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:56:33,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/ns/testMinorCompaction/11ddcd7bf8df7d1e6665f6c9cc6acd73/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-09 01:56:33,072 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:56:33,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 11ddcd7bf8df7d1e6665f6c9cc6acd73: 2023-03-09 01:56:33,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1678326940945.11ddcd7bf8df7d1e6665f6c9cc6acd73. 2023-03-09 01:56:33,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a4a8be322482491239e225079ce61b3a, disabling compactions & flushes 2023-03-09 01:56:33,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:56:33,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:56:33,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. after waiting 0 ms 2023-03-09 01:56:33,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:56:33,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/data/default/testMajorCompactionWithWeeklyPolicy/a4a8be322482491239e225079ce61b3a/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-09 01:56:33,077 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:56:33,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a4a8be322482491239e225079ce61b3a: 2023-03-09 01:56:33,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1678326864132.a4a8be322482491239e225079ce61b3a. 2023-03-09 01:56:33,191 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,46511,1678326819757; all regions closed. 2023-03-09 01:56:33,198 DEBUG [RS:0;jenkins-hbase4:46511] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/oldWALs 2023-03-09 01:56:33,198 INFO [RS:0;jenkins-hbase4:46511] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46511%2C1678326819757.meta:.meta(num 1678326821874) 2023-03-09 01:56:33,202 DEBUG [RS:0;jenkins-hbase4:46511] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/oldWALs 2023-03-09 01:56:33,202 INFO [RS:0;jenkins-hbase4:46511] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46511%2C1678326819757:(num 1678326821681) 2023-03-09 01:56:33,202 DEBUG [RS:0;jenkins-hbase4:46511] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:33,202 INFO [RS:0;jenkins-hbase4:46511] regionserver.LeaseManager(133): Closed leases 2023-03-09 01:56:33,202 INFO [RS:0;jenkins-hbase4:46511] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-09 01:56:33,202 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-09 01:56:33,203 INFO [RS:0;jenkins-hbase4:46511] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46511 2023-03-09 01:56:33,211 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-09 01:56:33,211 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46511,1678326819757 2023-03-09 01:56:33,211 ERROR [Listener at localhost/33973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@f5a9d99 rejected from java.util.concurrent.ThreadPoolExecutor@b6518aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-09 01:56:33,211 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46511,1678326819757] 2023-03-09 01:56:33,211 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-09 01:56:33,212 ERROR [Listener at localhost/33973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@240e3e17 rejected from java.util.concurrent.ThreadPoolExecutor@b6518aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-09 01:56:33,212 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46511,1678326819757; numProcessing=1 2023-03-09 01:56:33,213 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46511,1678326819757 already deleted, retry=false 2023-03-09 01:56:33,213 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46511,1678326819757 expired; onlineServers=0 2023-03-09 01:56:33,213 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39575,1678326818523' ***** 2023-03-09 01:56:33,213 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-09 01:56:33,214 DEBUG [M:0;jenkins-hbase4:39575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45ac5ade, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-09 01:56:33,214 INFO [M:0;jenkins-hbase4:39575] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39575,1678326818523 2023-03-09 01:56:33,214 INFO [M:0;jenkins-hbase4:39575] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39575,1678326818523; all regions closed. 2023-03-09 01:56:33,214 DEBUG [M:0;jenkins-hbase4:39575] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-09 01:56:33,214 DEBUG [M:0;jenkins-hbase4:39575] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-09 01:56:33,214 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-09 01:56:33,214 DEBUG [M:0;jenkins-hbase4:39575] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-09 01:56:33,214 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678326821353] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1678326821353,5,FailOnTimeoutGroup] 2023-03-09 01:56:33,214 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678326821353] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1678326821353,5,FailOnTimeoutGroup] 2023-03-09 01:56:33,215 INFO [M:0;jenkins-hbase4:39575] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-09 01:56:33,215 INFO [M:0;jenkins-hbase4:39575] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-09 01:56:33,215 INFO [M:0;jenkins-hbase4:39575] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-09 01:56:33,215 DEBUG [M:0;jenkins-hbase4:39575] master.HMaster(1512): Stopping service threads 2023-03-09 01:56:33,215 INFO [M:0;jenkins-hbase4:39575] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-09 01:56:33,216 ERROR [M:0;jenkins-hbase4:39575] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-09 01:56:33,216 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-09 01:56:33,216 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-09 01:56:33,216 INFO [M:0;jenkins-hbase4:39575] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-09 01:56:33,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-09 01:56:33,216 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-09 01:56:33,216 DEBUG [M:0;jenkins-hbase4:39575] zookeeper.ZKUtil(398): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-09 01:56:33,217 WARN [M:0;jenkins-hbase4:39575] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-09 01:56:33,217 INFO [M:0;jenkins-hbase4:39575] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-09 01:56:33,217 INFO [M:0;jenkins-hbase4:39575] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-09 01:56:33,217 DEBUG [M:0;jenkins-hbase4:39575] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-09 01:56:33,217 INFO [M:0;jenkins-hbase4:39575] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-09 01:56:33,217 DEBUG [M:0;jenkins-hbase4:39575] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-09 01:56:33,217 DEBUG [M:0;jenkins-hbase4:39575] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-09 01:56:33,217 DEBUG [M:0;jenkins-hbase4:39575] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-09 01:56:33,217 INFO [M:0;jenkins-hbase4:39575] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.78 KB heapSize=598.60 KB 2023-03-09 01:56:33,224 INFO [M:0;jenkins-hbase4:39575] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.78 KB at sequenceid=1006 (bloomFilter=true), to=hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5fd3c69ed1354c45ad646ba7d48e101e 2023-03-09 01:56:33,227 INFO [M:0;jenkins-hbase4:39575] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fd3c69ed1354c45ad646ba7d48e101e 2023-03-09 01:56:33,228 DEBUG [M:0;jenkins-hbase4:39575] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5fd3c69ed1354c45ad646ba7d48e101e as hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5fd3c69ed1354c45ad646ba7d48e101e 2023-03-09 01:56:33,231 INFO [M:0;jenkins-hbase4:39575] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fd3c69ed1354c45ad646ba7d48e101e 2023-03-09 01:56:33,231 INFO [M:0;jenkins-hbase4:39575] regionserver.HStore(1080): Added hdfs://localhost:45187/user/jenkins/test-data/effa58c5-a990-e825-3043-b7c16be32dbf/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5fd3c69ed1354c45ad646ba7d48e101e, entries=123, sequenceid=1006, filesize=33.3 K 2023-03-09 01:56:33,231 INFO [M:0;jenkins-hbase4:39575] regionserver.HRegion(2948): Finished flush of dataSize ~512.78 KB/525082, heapSize ~598.59 KB/612952, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 14ms, sequenceid=1006, compaction requested=false 2023-03-09 01:56:33,232 INFO [M:0;jenkins-hbase4:39575] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-09 01:56:33,232 DEBUG [M:0;jenkins-hbase4:39575] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-09 01:56:33,234 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-09 01:56:33,234 INFO [M:0;jenkins-hbase4:39575] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-09 01:56:33,235 INFO [M:0;jenkins-hbase4:39575] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39575 2023-03-09 01:56:33,237 DEBUG [M:0;jenkins-hbase4:39575] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39575,1678326818523 already deleted, retry=false 2023-03-09 01:56:33,313 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-09 01:56:33,313 INFO [RS:0;jenkins-hbase4:46511] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,46511,1678326819757; zookeeper connection closed. 2023-03-09 01:56:33,313 ERROR [Listener at localhost/33973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5ce02e54 rejected from java.util.concurrent.ThreadPoolExecutor@b6518aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-09 01:56:33,313 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): regionserver:46511-0x100c6facd450001, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-09 01:56:33,313 ERROR [Listener at localhost/33973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7be305b5 rejected from java.util.concurrent.ThreadPoolExecutor@b6518aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-09 01:56:33,313 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5df75bcd] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5df75bcd 2023-03-09 01:56:33,313 INFO [Listener at localhost/33973] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-09 01:56:33,413 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-09 01:56:33,413 INFO [M:0;jenkins-hbase4:39575] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39575,1678326818523; zookeeper connection closed. 2023-03-09 01:56:33,413 ERROR [Listener at localhost/33973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@367517bf rejected from java.util.concurrent.ThreadPoolExecutor@2254351a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-09 01:56:33,413 DEBUG [Listener at localhost/33973-EventThread] zookeeper.ZKWatcher(600): master:39575-0x100c6facd450000, quorum=127.0.0.1:50548, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-09 01:56:33,413 ERROR [Listener at localhost/33973-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@24f9c261 rejected from java.util.concurrent.ThreadPoolExecutor@2254351a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-09 01:56:33,415 WARN [Listener at localhost/33973] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-09 01:56:33,417 INFO [Listener at localhost/33973] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-09 01:56:33,524 WARN [BP-1290059279-172.31.14.131-1678326815090 heartbeating to localhost/127.0.0.1:45187] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-09 01:56:33,525 WARN [BP-1290059279-172.31.14.131-1678326815090 heartbeating to localhost/127.0.0.1:45187] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1290059279-172.31.14.131-1678326815090 (Datanode Uuid 80cf466a-c239-40a6-88c4-b2c3dde97f78) service to localhost/127.0.0.1:45187 2023-03-09 01:56:33,527 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/cluster_c7ae5eb2-da64-f932-cf16-ebb509eb1321/dfs/data/data1/current/BP-1290059279-172.31.14.131-1678326815090] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-09 01:56:33,527 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/c3c66e36-2b57-95a5-daec-36b69bbbd1cb/cluster_c7ae5eb2-da64-f932-cf16-ebb509eb1321/dfs/data/data2/current/BP-1290059279-172.31.14.131-1678326815090] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-09 01:56:33,551 INFO [Listener at localhost/33973] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-09 01:56:33,681 INFO [Listener at localhost/33973] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-09 01:56:33,760 INFO [Listener at localhost/33973] hbase.HBaseTestingUtility(1293): Minicluster is down